[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:03:10 dist-test-slave-6bbx krb5kdc[5516](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:03:10 dist-test-slave-6bbx krb5kdc[5516](info): set up 2 sockets
Aug 12 03:03:10 dist-test-slave-6bbx krb5kdc[5516](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 03:03:11.858892 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:03:11 dist-test-slave-6bbx krb5kdc[5516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:03:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:03:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:03:11.972235 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:45421
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:46293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:45421
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:12.093459 5538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:12.093683 5538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:12.093719 5538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:12.097826 5538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:03:12.097885 5538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:12.097904 5538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:12.097924 5538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:03:12.097939 5538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:03:12.103205 5538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:46293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:45421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:45421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:12.104573 5538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:12.105741 5538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:12.117806 5548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:12.119867 5547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:12.119974 5538 server_base.cc:1034] running on GCE node
W20260812 03:03:12.120162 5553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:12.120899 5538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:12.122670 5538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:12.123833 5538 hybrid_clock.cc:648] HybridClock initialized: now 1786503792123793 us; error 49 us; skew 500 ppm
Aug 12 03:03:12 dist-test-slave-6bbx krb5kdc[5516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503792, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:12.126621 5538 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:03:12.127801 5538 webserver.cc:460] Webserver started at http://127.5.96.190:44153/ using document root <none> and password file <none>
I20260812 03:03:12.128736 5538 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:12.128805 5538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:12.129184 5538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:12.131747 5538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "7ee9809cb63543639099fb51ef0c4294"
format_stamp: "Formatted at 2026-08-12 03:03:12 on dist-test-slave-6bbx"
server_key: "57af0d182f846c6867eb938a82017f21"
server_key_iv: "35aa8290578481b129d467dc3a78ddf2"
server_key_version: "encryptionkey@0"
I20260812 03:03:12.132272 5538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "7ee9809cb63543639099fb51ef0c4294"
format_stamp: "Formatted at 2026-08-12 03:03:12 on dist-test-slave-6bbx"
server_key: "57af0d182f846c6867eb938a82017f21"
server_key_iv: "35aa8290578481b129d467dc3a78ddf2"
server_key_version: "encryptionkey@0"
I20260812 03:03:12.137305 5538 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260812 03:03:12.140996 5563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:12.142624 5538 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:03:12.142879 5538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "7ee9809cb63543639099fb51ef0c4294"
format_stamp: "Formatted at 2026-08-12 03:03:12 on dist-test-slave-6bbx"
server_key: "57af0d182f846c6867eb938a82017f21"
server_key_iv: "35aa8290578481b129d467dc3a78ddf2"
server_key_version: "encryptionkey@0"
I20260812 03:03:12.142992 5538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:12.180011 5538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:12.183043 5538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:12.183249 5538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:12.192354 5538 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:45421
I20260812 03:03:12.192399 5646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:45421 every 8 connection(s)
I20260812 03:03:12.193701 5538 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:03:12.198237 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 5538
I20260812 03:03:12.198385 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:03:12.198988 5647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:12.199818 5506 external_mini_cluster.cc:1428] Setting key 7d85273205ae46424dc1b9a0a82b550b
I20260812 03:03:12.208591 5647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ee9809cb63543639099fb51ef0c4294: Bootstrap starting.
I20260812 03:03:12.212397 5647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ee9809cb63543639099fb51ef0c4294: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:12.213580 5647 log.cc:824] T 00000000000000000000000000000000 P 7ee9809cb63543639099fb51ef0c4294: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:12.216193 5647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ee9809cb63543639099fb51ef0c4294: No bootstrap required, opened a new log
Aug 12 03:03:12 dist-test-slave-6bbx krb5kdc[5516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:12.221436 5647 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ee9809cb63543639099fb51ef0c4294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ee9809cb63543639099fb51ef0c4294" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45421 } }
I20260812 03:03:12.221704 5647 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ee9809cb63543639099fb51ef0c4294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:12.221799 5647 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ee9809cb63543639099fb51ef0c4294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ee9809cb63543639099fb51ef0c4294, State: Initialized, Role: FOLLOWER
I20260812 03:03:12.222606 5647 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ee9809cb63543639099fb51ef0c4294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ee9809cb63543639099fb51ef0c4294" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45421 } }
I20260812 03:03:12.222808 5647 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ee9809cb63543639099fb51ef0c4294 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:03:12.222941 5647 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ee9809cb63543639099fb51ef0c4294 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:03:12.223104 5647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ee9809cb63543639099fb51ef0c4294 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:12.224152 5652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:12.204636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52470 (local address 127.5.96.190:45421)
0812 03:03:12.205060 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:12.205075 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:12.206166 (+ 1091us) server_negotiation.cc:407] Connection header received
0812 03:03:12.209351 (+ 3185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:12.209367 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:12.209751 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:12.210044 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:12.212322 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:12.213373 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:12.214713 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:12.215162 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:12.215307 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:12.219008 (+ 3701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:12.219051 (+ 43us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:12.219062 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:12.219089 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:12.220649 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:12.221140 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:12.221145 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:12.221149 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:12.221258 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:12.221576 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:12.221579 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:12.221580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:12.222129 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:12.222322 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:12.222900 (+ 578us) server_negotiation.cc:299] Negotiation successful
0812 03:03:12.223158 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":98,"threads_started":1}
I20260812 03:03:12.224306 5647 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ee9809cb63543639099fb51ef0c4294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ee9809cb63543639099fb51ef0c4294" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45421 } }
I20260812 03:03:12.224889 5647 leader_election.cc:302] T 00000000000000000000000000000000 P 7ee9809cb63543639099fb51ef0c4294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ee9809cb63543639099fb51ef0c4294; no voters:
I20260812 03:03:12.225272 5647 leader_election.cc:288] T 00000000000000000000000000000000 P 7ee9809cb63543639099fb51ef0c4294 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:03:12.225466 5657 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ee9809cb63543639099fb51ef0c4294 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:12.225667 5657 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ee9809cb63543639099fb51ef0c4294 [term 1 LEADER]: Becoming Leader. State: Replica: 7ee9809cb63543639099fb51ef0c4294, State: Running, Role: LEADER
I20260812 03:03:12.226084 5657 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ee9809cb63543639099fb51ef0c4294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ee9809cb63543639099fb51ef0c4294" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45421 } }
I20260812 03:03:12.227027 5647 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ee9809cb63543639099fb51ef0c4294 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:03:12.229291 5658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ee9809cb63543639099fb51ef0c4294 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ee9809cb63543639099fb51ef0c4294. Latest consensus state: current_term: 1 leader_uuid: "7ee9809cb63543639099fb51ef0c4294" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ee9809cb63543639099fb51ef0c4294" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45421 } } }
I20260812 03:03:12.229353 5660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ee9809cb63543639099fb51ef0c4294 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ee9809cb63543639099fb51ef0c4294" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ee9809cb63543639099fb51ef0c4294" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45421 } } }
I20260812 03:03:12.229431 5658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ee9809cb63543639099fb51ef0c4294 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:12.229465 5660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ee9809cb63543639099fb51ef0c4294 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:12.230522 5675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:03:12.236111 5675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:03:12.243844 5675 catalog_manager.cc:1347] Generated new cluster ID: 62328c9111804705ac303d0adfc83b12
I20260812 03:03:12.243908 5675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:03:12.275341 5675 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:03:12.276588 5675 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:03:12.285055 5675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ee9809cb63543639099fb51ef0c4294: Generated new TSK 0
I20260812 03:03:12.285841 5675 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.96.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
I20260812 03:03:12.314513 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:0
--local_ip_for_outbound_sockets=127.5.96.129
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:45421
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:46293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:12.477866 5691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:12.478119 5691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:12.478171 5691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:12.482265 5691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:12.482321 5691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:12.482398 5691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:03:12.488108 5691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:46293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:45421
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:12.489167 5691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:12.490176 5691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:12.501191 5698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:12.501266 5696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:12.501271 5701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:12.501607 5691 server_base.cc:1034] running on GCE node
I20260812 03:03:12.501948 5691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:12.502506 5691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:12.503688 5691 hybrid_clock.cc:648] HybridClock initialized: now 1786503792503646 us; error 32 us; skew 500 ppm
Aug 12 03:03:12 dist-test-slave-6bbx krb5kdc[5516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503792, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:12.506381 5691 init.cc:377] Logged in from keytab as kudu/127.5.96.129@KRBTEST.COM (short username kudu)
I20260812 03:03:12.507175 5691 webserver.cc:460] Webserver started at http://127.5.96.129:42067/ using document root <none> and password file <none>
I20260812 03:03:12.507751 5691 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:12.507838 5691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:12.508072 5691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:12.510165 5691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-0/data/instance:
uuid: "76fc0b3999f444e1b993e6c7f3e549c3"
format_stamp: "Formatted at 2026-08-12 03:03:12 on dist-test-slave-6bbx"
server_key: "e8fa0645350b24daef777cee94f2a84d"
server_key_iv: "d325407875b1228b681098865a9457dc"
server_key_version: "encryptionkey@0"
I20260812 03:03:12.510720 5691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance:
uuid: "76fc0b3999f444e1b993e6c7f3e549c3"
format_stamp: "Formatted at 2026-08-12 03:03:12 on dist-test-slave-6bbx"
server_key: "e8fa0645350b24daef777cee94f2a84d"
server_key_iv: "d325407875b1228b681098865a9457dc"
server_key_version: "encryptionkey@0"
I20260812 03:03:12.514436 5691 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260812 03:03:12.516770 5709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:12.517869 5691 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:03:12.517982 5691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "76fc0b3999f444e1b993e6c7f3e549c3"
format_stamp: "Formatted at 2026-08-12 03:03:12 on dist-test-slave-6bbx"
server_key: "e8fa0645350b24daef777cee94f2a84d"
server_key_iv: "d325407875b1228b681098865a9457dc"
server_key_version: "encryptionkey@0"
I20260812 03:03:12.518066 5691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:12.532639 5691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:12.535086 5691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:12.535243 5691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:12.535825 5691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:12.536895 5691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:12.536945 5691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:12.536996 5691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:12.537025 5691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:12.553709 5691 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:42939
I20260812 03:03:12.553915 5882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:42939 every 8 connection(s)
I20260812 03:03:12.554662 5691 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:03:12.561019 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 5691
I20260812 03:03:12.561137 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance
I20260812 03:03:12.561374 5506 external_mini_cluster.cc:1428] Setting key c2d02c6f1f210ef0c55d56c4bed88267
Aug 12 03:03:12 dist-test-slave-6bbx krb5kdc[5516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503792, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:12.570001 5652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:12.556916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:57313 (local address 127.5.96.190:45421)
0812 03:03:12.557057 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:12.557061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:12.558035 (+ 974us) server_negotiation.cc:407] Connection header received
0812 03:03:12.559910 (+ 1875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:12.559913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:12.559961 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:12.560033 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:12.561520 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:12.562061 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:12.563368 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:12.563597 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:12.563712 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:12.566789 (+ 3077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:12.566792 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:12.566794 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:12.566817 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:12.568082 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:12.568556 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:12.568559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:12.568560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:12.568635 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:12.568957 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:12.568961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:12.568962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:12.569144 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:12.569230 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:12.569704 (+ 474us) server_negotiation.cc:299] Negotiation successful
0812 03:03:12.569839 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 03:03:12.570808 5888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:12.557309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:45421 (local address 127.5.96.129:57313)
0812 03:03:12.557884 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:12.557927 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:12.559716 (+ 1789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:12.560175 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:12.560181 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:12.560670 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:12.561362 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:12.561380 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:12.562193 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:12.562196 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:12.563249 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:12.563259 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:12.563760 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:12.563765 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:12.563992 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:12.564648 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:12.564672 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:12.566628 (+ 1956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:12.568188 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:12.568194 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:12.568208 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:12.568461 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:12.568721 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:12.568724 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:12.568726 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:12.568859 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:12.569253 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:12.569259 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:12.569480 (+ 221us) client_negotiation.cc:769] Sending connection context
0812 03:03:12.569645 (+ 165us) client_negotiation.cc:240] Negotiation successful
0812 03:03:12.569949 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":192,"threads_started":1}
I20260812 03:03:12.572069 5883 heartbeater.cc:347] Connected to a master server at 127.5.96.190:45421
I20260812 03:03:12.572290 5883 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:12.572748 5883 heartbeater.cc:511] Master 127.5.96.190:45421 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.5.96.130@KRBTEST.COM; defaulting to no policy
I20260812 03:03:12.574960 5583 ts_manager.cc:194] Registered new tserver with Master: 76fc0b3999f444e1b993e6c7f3e549c3 (127.5.96.129:42939)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.130@KRBTEST.COM" created.
I20260812 03:03:12.576674 5583 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.129@KRBTEST.COM'} at 127.5.96.129:57313
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
I20260812 03:03:12.590339 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:0
--local_ip_for_outbound_sockets=127.5.96.130
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:45421
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:46293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:12.713734 5891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:12.713958 5891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:12.713999 5891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:12.718108 5891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:12.718155 5891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:12.718217 5891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:03:12.723205 5891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:46293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:45421
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:12.724174 5891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:12.725097 5891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:12.734195 5898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:12.734325 5897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:12.734206 5902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:12.734376 5891 server_base.cc:1034] running on GCE node
I20260812 03:03:12.734814 5891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:12.735368 5891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:12.736519 5891 hybrid_clock.cc:648] HybridClock initialized: now 1786503792736493 us; error 36 us; skew 500 ppm
Aug 12 03:03:12 dist-test-slave-6bbx krb5kdc[5516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503792, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:12.738993 5891 init.cc:377] Logged in from keytab as kudu/127.5.96.130@KRBTEST.COM (short username kudu)
I20260812 03:03:12.739830 5891 webserver.cc:460] Webserver started at http://127.5.96.130:39577/ using document root <none> and password file <none>
I20260812 03:03:12.740340 5891 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:12.740406 5891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:12.740605 5891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:12.742511 5891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-1/data/instance:
uuid: "8bbca980a1554cc69d4bb17e5e8430cb"
format_stamp: "Formatted at 2026-08-12 03:03:12 on dist-test-slave-6bbx"
server_key: "1af6b5e72b7ed93cd10d2480c1604fc5"
server_key_iv: "c6d9c5612af90905c9f82e777a3cef39"
server_key_version: "encryptionkey@0"
I20260812 03:03:12.743089 5891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance:
uuid: "8bbca980a1554cc69d4bb17e5e8430cb"
format_stamp: "Formatted at 2026-08-12 03:03:12 on dist-test-slave-6bbx"
server_key: "1af6b5e72b7ed93cd10d2480c1604fc5"
server_key_iv: "c6d9c5612af90905c9f82e777a3cef39"
server_key_version: "encryptionkey@0"
I20260812 03:03:12.746845 5891 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:03:12.749137 5913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:12.750247 5891 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:03:12.750350 5891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "8bbca980a1554cc69d4bb17e5e8430cb"
format_stamp: "Formatted at 2026-08-12 03:03:12 on dist-test-slave-6bbx"
server_key: "1af6b5e72b7ed93cd10d2480c1604fc5"
server_key_iv: "c6d9c5612af90905c9f82e777a3cef39"
server_key_version: "encryptionkey@0"
I20260812 03:03:12.750434 5891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:12.761761 5891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:12.764245 5891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:12.764395 5891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:12.764899 5891 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:12.765635 5891 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:12.765682 5891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:12.765730 5891 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:12.765758 5891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:12.774238 5891 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:35441
I20260812 03:03:12.774263 6084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:35441 every 8 connection(s)
I20260812 03:03:12.775060 5891 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:03:12.776597 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 5891
I20260812 03:03:12.776686 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance
I20260812 03:03:12.776949 5506 external_mini_cluster.cc:1428] Setting key 30dc9fcd0154f316fb270eaaeb4a65ef
Aug 12 03:03:12 dist-test-slave-6bbx krb5kdc[5516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503792, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.96.131@KRBTEST.COM; defaulting to no policy
I20260812 03:03:12.789413 5652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:12.776824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:49623 (local address 127.5.96.190:45421)
0812 03:03:12.776982 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:12.776987 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:12.777796 (+ 809us) server_negotiation.cc:407] Connection header received
0812 03:03:12.779349 (+ 1553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:12.779352 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:12.779397 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:12.779462 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:12.780899 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:12.781710 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:12.782831 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:12.783165 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:12.783286 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:12.785923 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:12.785926 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:12.785928 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:12.785951 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:12.787304 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:12.787923 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:12.787926 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:12.787927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:12.788000 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:12.788375 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:12.788377 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:12.788378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:12.788551 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:12.788625 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:12.789178 (+ 553us) server_negotiation.cc:299] Negotiation successful
0812 03:03:12.789294 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:03:12.790237 6088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:12.777185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:45421 (local address 127.5.96.130:49623)
0812 03:03:12.777664 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:12.777713 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:12.779101 (+ 1388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:12.779637 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:12.779645 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:12.780098 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:12.780727 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:12.780746 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:12.781823 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:12.781825 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:12.782699 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:12.782708 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:12.783293 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:12.783295 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:12.783432 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:12.784000 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:12.784015 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:12.785763 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:12.787425 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:12.787431 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:12.787445 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:12.787831 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:12.788083 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:12.788094 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:12.788096 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:12.788277 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:12.788673 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:12.788679 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:12.788942 (+ 263us) client_negotiation.cc:769] Sending connection context
0812 03:03:12.789139 (+ 197us) client_negotiation.cc:240] Negotiation successful
0812 03:03:12.789378 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":138,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.131@KRBTEST.COM" created.
I20260812 03:03:12.791330 6085 heartbeater.cc:347] Connected to a master server at 127.5.96.190:45421
I20260812 03:03:12.791535 6085 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:12.792027 6085 heartbeater.cc:511] Master 127.5.96.190:45421 requested a full tablet report, sending...
I20260812 03:03:12.793020 5583 ts_manager.cc:194] Registered new tserver with Master: 8bbca980a1554cc69d4bb17e5e8430cb (127.5.96.130:35441)
I20260812 03:03:12.793565 5583 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.130@KRBTEST.COM'} at 127.5.96.130:49623
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
I20260812 03:03:12.803766 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:0
--local_ip_for_outbound_sockets=127.5.96.131
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:45421
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:46293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:12.915433 6092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:12.915693 6092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:12.915755 6092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:12.919620 6092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:12.919691 6092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:12.919765 6092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:03:12.924919 6092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:46293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:45421
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:12.925856 6092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:12.926785 6092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:03:12.936154 6092 server_base.cc:1034] running on GCE node
W20260812 03:03:12.936092 6101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:12.936072 6099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:12.936075 6103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:12.936725 6092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:12.937325 6092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:12.938473 6092 hybrid_clock.cc:648] HybridClock initialized: now 1786503792938458 us; error 26 us; skew 500 ppm
Aug 12 03:03:12 dist-test-slave-6bbx krb5kdc[5516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503792, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:12.941115 6092 init.cc:377] Logged in from keytab as kudu/127.5.96.131@KRBTEST.COM (short username kudu)
I20260812 03:03:12.941911 6092 webserver.cc:460] Webserver started at http://127.5.96.131:38743/ using document root <none> and password file <none>
I20260812 03:03:12.942443 6092 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:12.942509 6092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:12.942703 6092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:12.944687 6092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-2/data/instance:
uuid: "b600d42096934ad194d87d854083bb8d"
format_stamp: "Formatted at 2026-08-12 03:03:12 on dist-test-slave-6bbx"
server_key: "8ee84ea3ddb7df50ba7d96e2b7aec91c"
server_key_iv: "460ddf881d440612220666e2f9664fa1"
server_key_version: "encryptionkey@0"
I20260812 03:03:12.945205 6092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance:
uuid: "b600d42096934ad194d87d854083bb8d"
format_stamp: "Formatted at 2026-08-12 03:03:12 on dist-test-slave-6bbx"
server_key: "8ee84ea3ddb7df50ba7d96e2b7aec91c"
server_key_iv: "460ddf881d440612220666e2f9664fa1"
server_key_version: "encryptionkey@0"
I20260812 03:03:12.948782 6092 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:03:12.951002 6114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:12.952149 6092 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:12.952261 6092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "b600d42096934ad194d87d854083bb8d"
format_stamp: "Formatted at 2026-08-12 03:03:12 on dist-test-slave-6bbx"
server_key: "8ee84ea3ddb7df50ba7d96e2b7aec91c"
server_key_iv: "460ddf881d440612220666e2f9664fa1"
server_key_version: "encryptionkey@0"
I20260812 03:03:12.952347 6092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:12.966452 6092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:12.968971 6092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:12.969130 6092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:12.969656 6092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:12.970422 6092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:12.970469 6092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:12.970520 6092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:12.970548 6092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:12.979035 6092 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:43415
I20260812 03:03:12.979068 6283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:43415 every 8 connection(s)
I20260812 03:03:12.979877 6092 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:03:12 dist-test-slave-6bbx krb5kdc[5516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503792, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:12.989581 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 6092
I20260812 03:03:12.989696 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance
I20260812 03:03:12.989959 5506 external_mini_cluster.cc:1428] Setting key a4c26489f79df57a9057bcc89d84e336
I20260812 03:03:12.992174 5652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:12.981565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:59709 (local address 127.5.96.190:45421)
0812 03:03:12.981739 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:12.981744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:12.982494 (+ 750us) server_negotiation.cc:407] Connection header received
0812 03:03:12.983773 (+ 1279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:12.983775 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:12.983823 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:12.983892 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:12.984915 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:12.985429 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:12.986447 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:12.986644 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:12.986711 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:12.988858 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:12.988861 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:12.988863 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:12.988884 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:12.990119 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:12.990632 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:12.990636 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:12.990638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:12.990720 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:12.991046 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:12.991049 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:12.991050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:12.991260 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:12.991362 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:12.991899 (+ 537us) server_negotiation.cc:299] Negotiation successful
0812 03:03:12.992020 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 03:03:12.992913 6287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:12.981917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:45421 (local address 127.5.96.131:59709)
0812 03:03:12.982352 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:12.982391 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:12.983545 (+ 1154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:12.984033 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:12.984039 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:12.984372 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:12.984806 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:12.984819 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:12.985521 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:12.985523 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:12.986363 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:12.986372 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:12.986740 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:12.986743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:12.986868 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:12.987360 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:12.987375 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:12.988716 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:12.990219 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:12.990225 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:12.990237 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:12.990537 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:12.990812 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:12.990815 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:12.990817 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:12.990943 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:12.991372 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:12.991378 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:12.991629 (+ 251us) client_negotiation.cc:769] Sending connection context
0812 03:03:12.991831 (+ 202us) client_negotiation.cc:240] Negotiation successful
0812 03:03:12.992117 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":132,"threads_started":1}
I20260812 03:03:12.993829 6284 heartbeater.cc:347] Connected to a master server at 127.5.96.190:45421
I20260812 03:03:12.994014 6284 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:12.994410 6284 heartbeater.cc:511] Master 127.5.96.190:45421 requested a full tablet report, sending...
I20260812 03:03:12.995250 5583 ts_manager.cc:194] Registered new tserver with Master: b600d42096934ad194d87d854083bb8d (127.5.96.131:43415)
I20260812 03:03:12.995808 5583 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.131@KRBTEST.COM'} at 127.5.96.131:59709
I20260812 03:03:13.003928 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 03:03:13 dist-test-slave-6bbx krb5kdc[5516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503793, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:03:13 dist-test-slave-6bbx krb5kdc[5516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503793, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.96.129@KRBTEST.COM
I20260812 03:03:13.030269 6295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:13.020912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43858 (local address 127.5.96.129:42939)
0812 03:03:13.021236 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:13.021251 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:13.021277 (+ 26us) server_negotiation.cc:407] Connection header received
0812 03:03:13.021333 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:13.021339 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:13.021512 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:13.021636 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:13.022340 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:13.023130 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:13.023928 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:13.024265 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:13.024345 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:13.026408 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:13.026417 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:13.026428 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:13.026451 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:13.028187 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:13.028705 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:13.028711 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:13.028713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:13.028811 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:13.029137 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:13.029141 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:13.029143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:13.029443 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:13.029663 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:13.029854 (+ 191us) server_negotiation.cc:299] Negotiation successful
0812 03:03:13.030059 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":116,"threads_started":1}
W20260812 03:03:13.031275 5759 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43858
Aug 12 03:03:13 dist-test-slave-6bbx krb5kdc[5516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:03:13 dist-test-slave-6bbx krb5kdc[5516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.129@KRBTEST.COM
I20260812 03:03:13.054944 6295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:13.048020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43862 (local address 127.5.96.129:42939)
0812 03:03:13.048157 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:13.048162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:13.048250 (+ 88us) server_negotiation.cc:407] Connection header received
0812 03:03:13.048356 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:13.048358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:13.048395 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:13.048457 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:13.049022 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:13.049499 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:13.050217 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:13.050420 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:13.050486 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:13.052554 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:13.052557 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:13.052559 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:13.052580 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:13.053662 (+ 1082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:13.054010 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:13.054012 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:13.054013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:13.054077 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:13.054277 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:13.054280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:13.054281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:13.054444 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:13.054538 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:13.054704 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 03:03:13.054797 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:03:13.055938 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 5691
I20260812 03:03:13.070490 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 5891
I20260812 03:03:13.089749 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 6092
I20260812 03:03:13.099769 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 5538
2026-08-12T03:03:13Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2427 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:03:13 dist-test-slave-6bbx krb5kdc[6311](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:03:13 dist-test-slave-6bbx krb5kdc[6311](info): set up 2 sockets
Aug 12 03:03:13 dist-test-slave-6bbx krb5kdc[6311](info): commencing operation
krb5kdc: starting...
W20260812 03:03:14.260941 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:03:14 dist-test-slave-6bbx krb5kdc[6311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:03:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:03:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:03:14.368475 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:35835
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:43563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:35835
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:14.475967 6331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:14.476203 6331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:14.476259 6331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:14.480206 6331 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:03:14.480254 6331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:14.480281 6331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:14.480304 6331 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:03:14.480331 6331 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:03:14.485355 6331 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:43563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:35835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:35835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:14.486265 6331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:14.487195 6331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:14.495045 6339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:14.495081 6338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:14.495102 6344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:14.495303 6331 server_base.cc:1034] running on GCE node
I20260812 03:03:14.495958 6331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:14.496958 6331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:14.498107 6331 hybrid_clock.cc:648] HybridClock initialized: now 1786503794498081 us; error 44 us; skew 500 ppm
Aug 12 03:03:14 dist-test-slave-6bbx krb5kdc[6311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503794, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:14.500659 6331 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:03:14.501441 6331 webserver.cc:460] Webserver started at http://127.5.96.190:41875/ using document root <none> and password file <none>
I20260812 03:03:14.501945 6331 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:14.502008 6331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:14.502224 6331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:14.504194 6331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "1eaa745988274adfb14eb45ed7c1d0c0"
format_stamp: "Formatted at 2026-08-12 03:03:14 on dist-test-slave-6bbx"
server_key: "e5d6f7eab41e2b5c8bf53590a2eae6ab"
server_key_iv: "6a8aceb7497c5cb548201bf102ce2c57"
server_key_version: "encryptionkey@0"
I20260812 03:03:14.504717 6331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "1eaa745988274adfb14eb45ed7c1d0c0"
format_stamp: "Formatted at 2026-08-12 03:03:14 on dist-test-slave-6bbx"
server_key: "e5d6f7eab41e2b5c8bf53590a2eae6ab"
server_key_iv: "6a8aceb7497c5cb548201bf102ce2c57"
server_key_version: "encryptionkey@0"
I20260812 03:03:14.508247 6331 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 03:03:14.510407 6353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:14.511475 6331 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:14.511584 6331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "1eaa745988274adfb14eb45ed7c1d0c0"
format_stamp: "Formatted at 2026-08-12 03:03:14 on dist-test-slave-6bbx"
server_key: "e5d6f7eab41e2b5c8bf53590a2eae6ab"
server_key_iv: "6a8aceb7497c5cb548201bf102ce2c57"
server_key_version: "encryptionkey@0"
I20260812 03:03:14.511701 6331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:14.544330 6331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:14.546710 6331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:14.546883 6331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:14.553400 6331 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:35835
I20260812 03:03:14.553428 6424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:35835 every 8 connection(s)
I20260812 03:03:14.554239 6331 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:03:14.556864 6425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:14.561976 6425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0: Bootstrap starting.
I20260812 03:03:14.563431 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 6331
I20260812 03:03:14.563521 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:03:14.563786 5506 external_mini_cluster.cc:1428] Setting key cffcddc09e340176a1df1fba88c0cc81
I20260812 03:03:14.564453 6425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:14.565241 6425 log.cc:824] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:14.567135 6425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0: No bootstrap required, opened a new log
I20260812 03:03:14.571100 6425 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eaa745988274adfb14eb45ed7c1d0c0" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35835 } }
I20260812 03:03:14.571378 6425 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:14.571439 6425 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1eaa745988274adfb14eb45ed7c1d0c0, State: Initialized, Role: FOLLOWER
I20260812 03:03:14.571920 6425 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eaa745988274adfb14eb45ed7c1d0c0" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35835 } }
I20260812 03:03:14.572065 6425 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:03:14.572114 6425 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:03:14.572211 6425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0 [term 0 FOLLOWER]: Advancing to term 1
Aug 12 03:03:14 dist-test-slave-6bbx krb5kdc[6311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:14.573022 6425 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eaa745988274adfb14eb45ed7c1d0c0" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35835 } }
I20260812 03:03:14.573385 6433 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:03:14.573380 6425 leader_election.cc:302] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1eaa745988274adfb14eb45ed7c1d0c0; no voters:
I20260812 03:03:14.573523 6433 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eaa745988274adfb14eb45ed7c1d0c0" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35835 } }
I20260812 03:03:14.573753 6425 leader_election.cc:288] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:03:14.574081 6435 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:14.574084 6433 leader_election.cc:302] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1eaa745988274adfb14eb45ed7c1d0c0; no voters:
I20260812 03:03:14.574368 6433 leader_election.cc:288] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers
I20260812 03:03:14.574555 6435 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0 [term 1 LEADER]: Becoming Leader. State: Replica: 1eaa745988274adfb14eb45ed7c1d0c0, State: Running, Role: LEADER
I20260812 03:03:14.574879 6435 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eaa745988274adfb14eb45ed7c1d0c0" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35835 } }
I20260812 03:03:14.574956 6425 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:03:14.576368 6436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1eaa745988274adfb14eb45ed7c1d0c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eaa745988274adfb14eb45ed7c1d0c0" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35835 } } }
I20260812 03:03:14.576488 6436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:14.576514 6433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1eaa745988274adfb14eb45ed7c1d0c0. Latest consensus state: current_term: 1 leader_uuid: "1eaa745988274adfb14eb45ed7c1d0c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eaa745988274adfb14eb45ed7c1d0c0" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35835 } } }
I20260812 03:03:14.576607 6433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:14.578269 6430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:14.564838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42226 (local address 127.5.96.190:35835)
0812 03:03:14.565243 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:14.565259 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:14.565291 (+ 32us) server_negotiation.cc:407] Connection header received
0812 03:03:14.566020 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:14.566040 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:14.566334 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:14.566587 (+ 253us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:14.567436 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:14.568439 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:14.569627 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:14.570061 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:14.570167 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:14.572928 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:14.572933 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:14.572945 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:14.572969 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:14.574821 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:14.575268 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:14.575274 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:14.575280 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:14.575391 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:14.575645 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:14.575648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:14.575650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:14.576182 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:14.576382 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:14.577093 (+ 711us) server_negotiation.cc:299] Negotiation successful
0812 03:03:14.577372 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":124,"threads_started":1}
W20260812 03:03:14.579896 6457 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:03:14.579958 6457 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:03:14.580014 6452 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:03:14.580718 6452 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:03:14.585680 6452 catalog_manager.cc:1347] Generated new cluster ID: d746691c4df340198c4b10001f674cf3
I20260812 03:03:14.585752 6452 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:03:14.605831 6452 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:03:14.606593 6452 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:03:14.615641 6452 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1eaa745988274adfb14eb45ed7c1d0c0: Generated new TSK 0
I20260812 03:03:14.616268 6452 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.96.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
I20260812 03:03:14.645921 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:0
--local_ip_for_outbound_sockets=127.5.96.129
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:35835
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:43563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:14.758423 6465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:14.758643 6465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:14.758682 6465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:14.762578 6465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:14.762625 6465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:14.762694 6465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:03:14.767577 6465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:43563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:35835
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:14.768465 6465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:14.769361 6465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:14.778263 6480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:14.778270 6477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:14.778478 6465 server_base.cc:1034] running on GCE node
W20260812 03:03:14.778275 6478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:14.778975 6465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:14.779506 6465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:14.780664 6465 hybrid_clock.cc:648] HybridClock initialized: now 1786503794780646 us; error 51 us; skew 500 ppm
Aug 12 03:03:14 dist-test-slave-6bbx krb5kdc[6311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503794, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:14.783262 6465 init.cc:377] Logged in from keytab as kudu/127.5.96.129@KRBTEST.COM (short username kudu)
I20260812 03:03:14.784096 6465 webserver.cc:460] Webserver started at http://127.5.96.129:38011/ using document root <none> and password file <none>
I20260812 03:03:14.784562 6465 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:14.784621 6465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:14.784780 6465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:14.786583 6465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-0/data/instance:
uuid: "47a524539cc64aa0af4e5cf40461c9d5"
format_stamp: "Formatted at 2026-08-12 03:03:14 on dist-test-slave-6bbx"
server_key: "4a0954793f62579988ee6db95fff02b4"
server_key_iv: "2ea126f8eca37af1c5110fe965a47d0e"
server_key_version: "encryptionkey@0"
I20260812 03:03:14.787065 6465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance:
uuid: "47a524539cc64aa0af4e5cf40461c9d5"
format_stamp: "Formatted at 2026-08-12 03:03:14 on dist-test-slave-6bbx"
server_key: "4a0954793f62579988ee6db95fff02b4"
server_key_iv: "2ea126f8eca37af1c5110fe965a47d0e"
server_key_version: "encryptionkey@0"
I20260812 03:03:14.790429 6465 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 03:03:14.792671 6490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:14.793741 6465 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:03:14.793836 6465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "47a524539cc64aa0af4e5cf40461c9d5"
format_stamp: "Formatted at 2026-08-12 03:03:14 on dist-test-slave-6bbx"
server_key: "4a0954793f62579988ee6db95fff02b4"
server_key_iv: "2ea126f8eca37af1c5110fe965a47d0e"
server_key_version: "encryptionkey@0"
I20260812 03:03:14.793915 6465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:14.823096 6465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:14.825847 6465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:14.826010 6465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:14.826551 6465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:14.827355 6465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:14.827402 6465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:14.827440 6465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:14.827461 6465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:14.835645 6465 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:36137
I20260812 03:03:14.835692 6655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:36137 every 8 connection(s)
I20260812 03:03:14.836483 6465 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:03:14.841871 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 6465
I20260812 03:03:14.841987 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance
I20260812 03:03:14.842240 5506 external_mini_cluster.cc:1428] Setting key 60237e5315487db3a2c4479375d5289e
Aug 12 03:03:14 dist-test-slave-6bbx krb5kdc[6311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503794, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:14.850225 6430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:14.838118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:43295 (local address 127.5.96.190:35835)
0812 03:03:14.838240 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:14.838244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:14.839032 (+ 788us) server_negotiation.cc:407] Connection header received
0812 03:03:14.840663 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:14.840666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:14.840723 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:14.840796 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:14.841996 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:14.842586 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:14.844027 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:14.844337 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:14.844443 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:14.846820 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:14.846823 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:14.846825 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:14.846849 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:14.848157 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:14.848758 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:14.848761 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:14.848762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:14.848841 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:14.849200 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:14.849202 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:14.849203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:14.849364 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:14.849434 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:14.850010 (+ 576us) server_negotiation.cc:299] Negotiation successful
0812 03:03:14.850107 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:03:14.851322 6661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:14.838452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:35835 (local address 127.5.96.129:43295)
0812 03:03:14.838895 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:14.838936 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:14.840435 (+ 1499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:14.840958 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:14.840965 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:14.841297 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:14.841804 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:14.841827 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:14.842712 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:14.842715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:14.843907 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:14.843919 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:14.844463 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:14.844465 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:14.844614 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:14.845101 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:14.845116 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:14.846664 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:14.848277 (+ 1613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:14.848284 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:14.848299 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:14.848661 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:14.848925 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:14.848928 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:14.848930 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:14.849117 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:14.849464 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:14.849471 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:14.849779 (+ 308us) client_negotiation.cc:769] Sending connection context
0812 03:03:14.849993 (+ 214us) client_negotiation.cc:240] Negotiation successful
0812 03:03:14.850314 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":114,"threads_started":1}
I20260812 03:03:14.852516 6656 heartbeater.cc:347] Connected to a master server at 127.5.96.190:35835
I20260812 03:03:14.852736 6656 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.5.96.130@KRBTEST.COM; defaulting to no policy
I20260812 03:03:14.853197 6656 heartbeater.cc:511] Master 127.5.96.190:35835 requested a full tablet report, sending...
I20260812 03:03:14.854645 6376 ts_manager.cc:194] Registered new tserver with Master: 47a524539cc64aa0af4e5cf40461c9d5 (127.5.96.129:36137)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.130@KRBTEST.COM" created.
I20260812 03:03:14.855814 6376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.129@KRBTEST.COM'} at 127.5.96.129:43295
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
I20260812 03:03:14.868106 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:0
--local_ip_for_outbound_sockets=127.5.96.130
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:35835
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:43563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:14.974879 6665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:14.975143 6665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:14.975193 6665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:14.979179 6665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:14.979226 6665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:14.979290 6665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:03:14.984282 6665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:43563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:35835
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:14.985138 6665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:14.986013 6665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:14.994812 6672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:14.994848 6675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:14.994958 6665 server_base.cc:1034] running on GCE node
W20260812 03:03:14.994812 6673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:14.995419 6665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:14.996006 6665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:14.997175 6665 hybrid_clock.cc:648] HybridClock initialized: now 1786503794997157 us; error 30 us; skew 500 ppm
Aug 12 03:03:14 dist-test-slave-6bbx krb5kdc[6311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503794, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:14.999732 6665 init.cc:377] Logged in from keytab as kudu/127.5.96.130@KRBTEST.COM (short username kudu)
I20260812 03:03:15.000476 6665 webserver.cc:460] Webserver started at http://127.5.96.130:45335/ using document root <none> and password file <none>
I20260812 03:03:15.000964 6665 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:15.001027 6665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:15.001220 6665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:15.003115 6665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-1/data/instance:
uuid: "6aedbb883a914058a857dc57c0c36476"
format_stamp: "Formatted at 2026-08-12 03:03:14 on dist-test-slave-6bbx"
server_key: "3bd53affd25ae27cc31ad861ab2d8bc5"
server_key_iv: "7ddf9dc59b3c6907bacc69bfac269ac8"
server_key_version: "encryptionkey@0"
I20260812 03:03:15.003657 6665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance:
uuid: "6aedbb883a914058a857dc57c0c36476"
format_stamp: "Formatted at 2026-08-12 03:03:14 on dist-test-slave-6bbx"
server_key: "3bd53affd25ae27cc31ad861ab2d8bc5"
server_key_iv: "7ddf9dc59b3c6907bacc69bfac269ac8"
server_key_version: "encryptionkey@0"
I20260812 03:03:15.007011 6665 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:03:15.009204 6685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:15.010273 6665 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:03:15.010375 6665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "6aedbb883a914058a857dc57c0c36476"
format_stamp: "Formatted at 2026-08-12 03:03:14 on dist-test-slave-6bbx"
server_key: "3bd53affd25ae27cc31ad861ab2d8bc5"
server_key_iv: "7ddf9dc59b3c6907bacc69bfac269ac8"
server_key_version: "encryptionkey@0"
I20260812 03:03:15.010461 6665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:15.036574 6665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:15.039393 6665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:15.039548 6665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:15.040081 6665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:15.040824 6665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:15.040870 6665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:15.040910 6665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:15.040939 6665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:15.048906 6665 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:35967
I20260812 03:03:15.048966 6853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:35967 every 8 connection(s)
I20260812 03:03:15.049705 6665 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:03:15.053617 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 6665
I20260812 03:03:15.053737 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance
I20260812 03:03:15.053975 5506 external_mini_cluster.cc:1428] Setting key 11ff10d5f870c856e930f24b8107a1ef
Aug 12 03:03:15 dist-test-slave-6bbx krb5kdc[6311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503794, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:15.062570 6430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:15.051316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:56249 (local address 127.5.96.190:35835)
0812 03:03:15.051439 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:15.051443 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:15.052205 (+ 762us) server_negotiation.cc:407] Connection header received
0812 03:03:15.053410 (+ 1205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:15.053413 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:15.053471 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:15.053562 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:15.054726 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.055320 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.056422 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.056634 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.056720 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:15.059259 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:15.059263 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:15.059264 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:15.059289 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:15.060549 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:15.061131 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:15.061133 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:15.061134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:15.061201 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:15.061561 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:15.061563 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:15.061564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:15.061714 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:15.061783 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:15.062354 (+ 571us) server_negotiation.cc:299] Negotiation successful
0812 03:03:15.062454 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:03:15.063601 6857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:15.051635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:35835 (local address 127.5.96.130:56249)
0812 03:03:15.052068 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:15.052107 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:15.053231 (+ 1124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:15.053694 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:15.053700 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:15.054076 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:15.054591 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:15.054607 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.055434 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.055437 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:15.056309 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:15.056319 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.056770 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.056774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:15.056926 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:15.057418 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:15.057437 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:15.059056 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:15.060662 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:15.060669 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:15.060683 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:15.061023 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:15.061289 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:15.061292 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:15.061294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:15.061475 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:15.061808 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:15.061814 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:15.062121 (+ 307us) client_negotiation.cc:769] Sending connection context
0812 03:03:15.062330 (+ 209us) client_negotiation.cc:240] Negotiation successful
0812 03:03:15.062640 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":106,"threads_started":1}
WARNING: no policy specified for kudu/127.5.96.131@KRBTEST.COM; defaulting to no policy
I20260812 03:03:15.064707 6855 heartbeater.cc:347] Connected to a master server at 127.5.96.190:35835
I20260812 03:03:15.064917 6855 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:15.065343 6855 heartbeater.cc:511] Master 127.5.96.190:35835 requested a full tablet report, sending...
I20260812 03:03:15.066273 6376 ts_manager.cc:194] Registered new tserver with Master: 6aedbb883a914058a857dc57c0c36476 (127.5.96.130:35967)
I20260812 03:03:15.066804 6376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.130@KRBTEST.COM'} at 127.5.96.130:56249
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
I20260812 03:03:15.081701 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:0
--local_ip_for_outbound_sockets=127.5.96.131
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:35835
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:43563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:15.187602 6864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:15.187955 6864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:15.188062 6864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:15.192028 6864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:15.192072 6864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:15.192148 6864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:03:15.197162 6864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:43563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:35835
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:15.198045 6864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:15.198932 6864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:15.207784 6871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:15.207850 6864 server_base.cc:1034] running on GCE node
W20260812 03:03:15.207792 6876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:15.207792 6872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:15.208364 6864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:15.208895 6864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:15.210047 6864 hybrid_clock.cc:648] HybridClock initialized: now 1786503795210032 us; error 24 us; skew 500 ppm
Aug 12 03:03:15 dist-test-slave-6bbx krb5kdc[6311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503795, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:15.212558 6864 init.cc:377] Logged in from keytab as kudu/127.5.96.131@KRBTEST.COM (short username kudu)
I20260812 03:03:15.213356 6864 webserver.cc:460] Webserver started at http://127.5.96.131:33243/ using document root <none> and password file <none>
I20260812 03:03:15.213836 6864 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:15.213901 6864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:15.214125 6864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:15.216144 6864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-2/data/instance:
uuid: "83d4e33f4c7d49f193473b07acd4399e"
format_stamp: "Formatted at 2026-08-12 03:03:15 on dist-test-slave-6bbx"
server_key: "5ecce8445697be27d987425c18bd9995"
server_key_iv: "bb048d8de8487c1591a273c8d8cb95f1"
server_key_version: "encryptionkey@0"
I20260812 03:03:15.216710 6864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance:
uuid: "83d4e33f4c7d49f193473b07acd4399e"
format_stamp: "Formatted at 2026-08-12 03:03:15 on dist-test-slave-6bbx"
server_key: "5ecce8445697be27d987425c18bd9995"
server_key_iv: "bb048d8de8487c1591a273c8d8cb95f1"
server_key_version: "encryptionkey@0"
I20260812 03:03:15.220086 6864 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:03:15.222271 6885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:15.223371 6864 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:15.223475 6864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "83d4e33f4c7d49f193473b07acd4399e"
format_stamp: "Formatted at 2026-08-12 03:03:15 on dist-test-slave-6bbx"
server_key: "5ecce8445697be27d987425c18bd9995"
server_key_iv: "bb048d8de8487c1591a273c8d8cb95f1"
server_key_version: "encryptionkey@0"
I20260812 03:03:15.223560 6864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:15.248426 6864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:15.250780 6864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:15.250934 6864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:15.251449 6864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:15.252209 6864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:15.252256 6864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:15.252305 6864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:15.252334 6864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:15.260478 6864 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:35433
I20260812 03:03:15.260501 7062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:35433 every 8 connection(s)
I20260812 03:03:15.261246 6864 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:03:15.267058 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 6864
I20260812 03:03:15.267174 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance
I20260812 03:03:15.267410 5506 external_mini_cluster.cc:1428] Setting key 74e6c26e7cbd940df3ad68763297b3bf
Aug 12 03:03:15 dist-test-slave-6bbx krb5kdc[6311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503795, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:15.273784 6430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:15.262840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:57039 (local address 127.5.96.190:35835)
0812 03:03:15.262960 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:15.262964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:15.263745 (+ 781us) server_negotiation.cc:407] Connection header received
0812 03:03:15.265003 (+ 1258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:15.265006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:15.265064 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:15.265134 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:15.266192 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.266711 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.267913 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.268227 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.268325 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:15.270622 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:15.270625 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:15.270627 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:15.270650 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:15.271927 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:15.272401 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:15.272403 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:15.272405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:15.272480 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:15.272797 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:15.272800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:15.272802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:15.272979 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:15.273064 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:15.273512 (+ 448us) server_negotiation.cc:299] Negotiation successful
0812 03:03:15.273629 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:03:15.274444 7069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:15.263166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:35835 (local address 127.5.96.131:57039)
0812 03:03:15.263584 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:15.263623 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:15.264802 (+ 1179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:15.265311 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:15.265318 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:15.265631 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:15.266049 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:15.266062 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.266835 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.266838 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:15.267799 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:15.267810 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.268394 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.268398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:15.268570 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:15.269064 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:15.269080 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:15.270467 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:15.272031 (+ 1564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:15.272037 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:15.272051 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:15.272306 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:15.272562 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:15.272565 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:15.272567 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:15.272709 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:15.273076 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:15.273083 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:15.273299 (+ 216us) client_negotiation.cc:769] Sending connection context
0812 03:03:15.273452 (+ 153us) client_negotiation.cc:240] Negotiation successful
0812 03:03:15.273731 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":116,"threads_started":1}
I20260812 03:03:15.275374 7063 heartbeater.cc:347] Connected to a master server at 127.5.96.190:35835
I20260812 03:03:15.275576 7063 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:15.276014 7063 heartbeater.cc:511] Master 127.5.96.190:35835 requested a full tablet report, sending...
I20260812 03:03:15.276990 6376 ts_manager.cc:194] Registered new tserver with Master: 83d4e33f4c7d49f193473b07acd4399e (127.5.96.131:35433)
I20260812 03:03:15.277493 6376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.131@KRBTEST.COM'} at 127.5.96.131:57039
I20260812 03:03:15.281087 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:03:15.290954 6430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:15.285108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42236 (local address 127.5.96.190:35835)
0812 03:03:15.285234 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:15.285237 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:15.285365 (+ 128us) server_negotiation.cc:407] Connection header received
0812 03:03:15.285452 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:15.285454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:15.285487 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:15.285545 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:15.286130 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.286588 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.287294 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.287485 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.287548 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:15.288635 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:15.288638 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:15.288639 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:15.288659 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:15.289790 (+ 1131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:15.290085 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:15.290087 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:15.290089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:15.290151 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:15.290319 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:15.290320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:15.290322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:15.290498 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:15.290595 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:15.290735 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 03:03:15.290817 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 03:03:15.296185 6376 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42236:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:03:15.299556 6376 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:03:15.314218 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:15.308730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:36137 (local address 127.0.0.1:42476)
0812 03:03:15.309528 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:15.309541 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:15.309626 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:15.310116 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:15.310119 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:15.310137 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:15.310225 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:15.310231 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.311146 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.311149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:15.312615 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:15.312623 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.313578 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.313582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:15.313705 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:15.313749 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 03:03:15.313948 (+ 199us) client_negotiation.cc:240] Negotiation successful
0812 03:03:15.314074 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":750,"mutex_wait_us":258,"thread_start_us":92,"threads_started":1}
I20260812 03:03:15.314333 7089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:15.309022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58906 (local address 127.5.96.131:35433)
0812 03:03:15.309434 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:15.309446 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:15.309467 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:03:15.309533 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:15.309539 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:15.309724 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:15.310193 (+ 469us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:15.310332 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.311431 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.312947 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.313774 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.313884 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:15.314017 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:15.314117 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 03:03:15.314199 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":115,"threads_started":1}
I20260812 03:03:15.314278 7083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:15.308549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:35433 (local address 127.0.0.1:58906)
0812 03:03:15.308919 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:15.308968 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:15.309097 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:15.309897 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:15.309903 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:15.309926 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:15.310045 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:15.310052 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.311567 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.311571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:15.312832 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:15.312842 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.313885 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.313889 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:15.313976 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:15.314000 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:03:15.314073 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 03:03:15.314141 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":107,"spinlock_wait_cycles":4352,"thread_start_us":68,"threads_started":1}
I20260812 03:03:15.314672 7090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:15.309129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42476 (local address 127.5.96.129:36137)
0812 03:03:15.309404 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:15.309419 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:15.309718 (+ 299us) server_negotiation.cc:407] Connection header received
0812 03:03:15.309801 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:15.309807 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:15.309991 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:15.310127 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:15.310325 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.311029 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.312748 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.313458 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.313564 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:15.313775 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:15.314374 (+ 599us) server_negotiation.cc:299] Negotiation successful
0812 03:03:15.314467 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":105,"threads_started":1}
I20260812 03:03:15.315495 7087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:15.308332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:35967 (local address 127.0.0.1:60546)
0812 03:03:15.308921 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:15.308965 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:15.309097 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:15.310179 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:15.310182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:15.310200 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:15.310292 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:15.310299 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.311915 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.311918 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:15.313259 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:15.313271 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.314360 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.314364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:15.314452 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:15.314474 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:03:15.314523 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 03:03:15.315366 (+ 843us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":81,"spinlock_wait_cycles":6912,"thread_start_us":62,"threads_started":1}
I20260812 03:03:15.316277 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:15.308425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60546 (local address 127.5.96.130:35967)
0812 03:03:15.309755 (+ 1330us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:15.309766 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:15.309789 (+ 23us) server_negotiation.cc:407] Connection header received
0812 03:03:15.309860 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:15.309864 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:15.310037 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:15.310436 (+ 399us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:15.310593 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.311657 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.313384 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.314184 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.315839 (+ 1655us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:15.315975 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:15.316028 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 03:03:15.316123 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1214,"thread_start_us":106,"threads_started":1}
I20260812 03:03:15.317695 6554 tablet_service.cc:1467] Processing CreateTablet for tablet 85e2137d9a3c4901be7acb75778099ac (DEFAULT_TABLE table=test-table [id=53274ad5400d4d58b5f45fc41f5bdfbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:15.318754 6554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85e2137d9a3c4901be7acb75778099ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:15.318687 6760 tablet_service.cc:1467] Processing CreateTablet for tablet 85e2137d9a3c4901be7acb75778099ac (DEFAULT_TABLE table=test-table [id=53274ad5400d4d58b5f45fc41f5bdfbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:15.319722 6760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85e2137d9a3c4901be7acb75778099ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:15.319554 6954 tablet_service.cc:1467] Processing CreateTablet for tablet 85e2137d9a3c4901be7acb75778099ac (DEFAULT_TABLE table=test-table [id=53274ad5400d4d58b5f45fc41f5bdfbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:15.320587 6954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85e2137d9a3c4901be7acb75778099ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:15.325415 7092 tablet_bootstrap.cc:492] T 85e2137d9a3c4901be7acb75778099ac P 47a524539cc64aa0af4e5cf40461c9d5: Bootstrap starting.
I20260812 03:03:15.327225 7095 tablet_bootstrap.cc:492] T 85e2137d9a3c4901be7acb75778099ac P 83d4e33f4c7d49f193473b07acd4399e: Bootstrap starting.
I20260812 03:03:15.327924 7092 tablet_bootstrap.cc:654] T 85e2137d9a3c4901be7acb75778099ac P 47a524539cc64aa0af4e5cf40461c9d5: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:15.327979 7093 tablet_bootstrap.cc:492] T 85e2137d9a3c4901be7acb75778099ac P 6aedbb883a914058a857dc57c0c36476: Bootstrap starting.
I20260812 03:03:15.329481 7095 tablet_bootstrap.cc:654] T 85e2137d9a3c4901be7acb75778099ac P 83d4e33f4c7d49f193473b07acd4399e: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:15.329586 7092 log.cc:824] T 85e2137d9a3c4901be7acb75778099ac P 47a524539cc64aa0af4e5cf40461c9d5: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:15.330107 7093 tablet_bootstrap.cc:654] T 85e2137d9a3c4901be7acb75778099ac P 6aedbb883a914058a857dc57c0c36476: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:15.330660 7095 log.cc:824] T 85e2137d9a3c4901be7acb75778099ac P 83d4e33f4c7d49f193473b07acd4399e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:15.330803 7093 log.cc:824] T 85e2137d9a3c4901be7acb75778099ac P 6aedbb883a914058a857dc57c0c36476: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:15.331692 7092 tablet_bootstrap.cc:492] T 85e2137d9a3c4901be7acb75778099ac P 47a524539cc64aa0af4e5cf40461c9d5: No bootstrap required, opened a new log
I20260812 03:03:15.331878 7092 ts_tablet_manager.cc:1397] T 85e2137d9a3c4901be7acb75778099ac P 47a524539cc64aa0af4e5cf40461c9d5: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260812 03:03:15.332543 7095 tablet_bootstrap.cc:492] T 85e2137d9a3c4901be7acb75778099ac P 83d4e33f4c7d49f193473b07acd4399e: No bootstrap required, opened a new log
I20260812 03:03:15.332543 7093 tablet_bootstrap.cc:492] T 85e2137d9a3c4901be7acb75778099ac P 6aedbb883a914058a857dc57c0c36476: No bootstrap required, opened a new log
I20260812 03:03:15.332722 7095 ts_tablet_manager.cc:1397] T 85e2137d9a3c4901be7acb75778099ac P 83d4e33f4c7d49f193473b07acd4399e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:03:15.332731 7093 ts_tablet_manager.cc:1397] T 85e2137d9a3c4901be7acb75778099ac P 6aedbb883a914058a857dc57c0c36476: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 03:03:15.334913 7092 raft_consensus.cc:348] T 85e2137d9a3c4901be7acb75778099ac P 47a524539cc64aa0af4e5cf40461c9d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aedbb883a914058a857dc57c0c36476" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35967 } } peers { permanent_uuid: "83d4e33f4c7d49f193473b07acd4399e" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 35433 } } peers { permanent_uuid: "47a524539cc64aa0af4e5cf40461c9d5" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36137 } }
I20260812 03:03:15.335127 7092 raft_consensus.cc:374] T 85e2137d9a3c4901be7acb75778099ac P 47a524539cc64aa0af4e5cf40461c9d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:15.335180 7092 raft_consensus.cc:729] T 85e2137d9a3c4901be7acb75778099ac P 47a524539cc64aa0af4e5cf40461c9d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47a524539cc64aa0af4e5cf40461c9d5, State: Initialized, Role: FOLLOWER
I20260812 03:03:15.335570 7092 consensus_queue.cc:260] T 85e2137d9a3c4901be7acb75778099ac P 47a524539cc64aa0af4e5cf40461c9d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aedbb883a914058a857dc57c0c36476" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35967 } } peers { permanent_uuid: "83d4e33f4c7d49f193473b07acd4399e" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 35433 } } peers { permanent_uuid: "47a524539cc64aa0af4e5cf40461c9d5" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36137 } }
I20260812 03:03:15.336290 6656 heartbeater.cc:503] Master 127.5.96.190:35835 was elected leader, sending a full tablet report...
I20260812 03:03:15.336494 7092 ts_tablet_manager.cc:1428] T 85e2137d9a3c4901be7acb75778099ac P 47a524539cc64aa0af4e5cf40461c9d5: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260812 03:03:15.336356 7093 raft_consensus.cc:348] T 85e2137d9a3c4901be7acb75778099ac P 6aedbb883a914058a857dc57c0c36476 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aedbb883a914058a857dc57c0c36476" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35967 } } peers { permanent_uuid: "83d4e33f4c7d49f193473b07acd4399e" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 35433 } } peers { permanent_uuid: "47a524539cc64aa0af4e5cf40461c9d5" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36137 } }
I20260812 03:03:15.336607 7093 raft_consensus.cc:374] T 85e2137d9a3c4901be7acb75778099ac P 6aedbb883a914058a857dc57c0c36476 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:15.336668 7093 raft_consensus.cc:729] T 85e2137d9a3c4901be7acb75778099ac P 6aedbb883a914058a857dc57c0c36476 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6aedbb883a914058a857dc57c0c36476, State: Initialized, Role: FOLLOWER
I20260812 03:03:15.336936 7095 raft_consensus.cc:348] T 85e2137d9a3c4901be7acb75778099ac P 83d4e33f4c7d49f193473b07acd4399e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aedbb883a914058a857dc57c0c36476" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35967 } } peers { permanent_uuid: "83d4e33f4c7d49f193473b07acd4399e" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 35433 } } peers { permanent_uuid: "47a524539cc64aa0af4e5cf40461c9d5" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36137 } }
I20260812 03:03:15.337152 7095 raft_consensus.cc:374] T 85e2137d9a3c4901be7acb75778099ac P 83d4e33f4c7d49f193473b07acd4399e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:15.337091 7093 consensus_queue.cc:260] T 85e2137d9a3c4901be7acb75778099ac P 6aedbb883a914058a857dc57c0c36476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aedbb883a914058a857dc57c0c36476" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35967 } } peers { permanent_uuid: "83d4e33f4c7d49f193473b07acd4399e" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 35433 } } peers { permanent_uuid: "47a524539cc64aa0af4e5cf40461c9d5" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36137 } }
I20260812 03:03:15.337208 7095 raft_consensus.cc:729] T 85e2137d9a3c4901be7acb75778099ac P 83d4e33f4c7d49f193473b07acd4399e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83d4e33f4c7d49f193473b07acd4399e, State: Initialized, Role: FOLLOWER
W20260812 03:03:15.337553 6659 tablet.cc:2369] T 85e2137d9a3c4901be7acb75778099ac P 47a524539cc64aa0af4e5cf40461c9d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:03:15.337636 7095 consensus_queue.cc:260] T 85e2137d9a3c4901be7acb75778099ac P 83d4e33f4c7d49f193473b07acd4399e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aedbb883a914058a857dc57c0c36476" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35967 } } peers { permanent_uuid: "83d4e33f4c7d49f193473b07acd4399e" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 35433 } } peers { permanent_uuid: "47a524539cc64aa0af4e5cf40461c9d5" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36137 } }
I20260812 03:03:15.338025 7093 ts_tablet_manager.cc:1428] T 85e2137d9a3c4901be7acb75778099ac P 6aedbb883a914058a857dc57c0c36476: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 03:03:15.338343 6855 heartbeater.cc:503] Master 127.5.96.190:35835 was elected leader, sending a full tablet report...
I20260812 03:03:15.339244 7095 ts_tablet_manager.cc:1428] T 85e2137d9a3c4901be7acb75778099ac P 83d4e33f4c7d49f193473b07acd4399e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 03:03:15.339694 7063 heartbeater.cc:503] Master 127.5.96.190:35835 was elected leader, sending a full tablet report...
I20260812 03:03:15.402994 7102 raft_consensus.cc:482] T 85e2137d9a3c4901be7acb75778099ac P 47a524539cc64aa0af4e5cf40461c9d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:03:15.403165 7102 raft_consensus.cc:504] T 85e2137d9a3c4901be7acb75778099ac P 47a524539cc64aa0af4e5cf40461c9d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aedbb883a914058a857dc57c0c36476" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35967 } } peers { permanent_uuid: "83d4e33f4c7d49f193473b07acd4399e" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 35433 } } peers { permanent_uuid: "47a524539cc64aa0af4e5cf40461c9d5" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36137 } }
I20260812 03:03:15.404268 7102 leader_election.cc:288] T 85e2137d9a3c4901be7acb75778099ac P 47a524539cc64aa0af4e5cf40461c9d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6aedbb883a914058a857dc57c0c36476 (127.5.96.130:35967), 83d4e33f4c7d49f193473b07acd4399e (127.5.96.131:35433)
I20260812 03:03:15.409056 7089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:15.404548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:58607 (local address 127.5.96.131:35433)
0812 03:03:15.404667 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:15.404671 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:15.404959 (+ 288us) server_negotiation.cc:407] Connection header received
0812 03:03:15.405032 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:15.405034 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:15.405076 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:15.405151 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:15.405444 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.406225 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.407914 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.408498 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.408616 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:15.408666 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:15.408868 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 03:03:15.408934 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:03:15.409056 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:15.404839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:55957 (local address 127.5.96.130:35967)
0812 03:03:15.404955 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:15.404958 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:15.404972 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:03:15.405022 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:15.405025 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:15.405073 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:15.405152 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:15.405560 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.406278 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.407901 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.408505 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.408616 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:15.408666 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:15.408868 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 03:03:15.408933 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:03:15.409193 7107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:15.404417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:35967 (local address 127.5.96.129:55957)
0812 03:03:15.404742 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:15.404768 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:15.404879 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:15.405294 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:15.405297 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:15.405316 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:15.405437 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:15.405445 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.406383 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.406386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:15.407796 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:15.407807 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.408608 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.408612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:15.408768 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:15.408786 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 03:03:15.408847 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 03:03:15.409016 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":132,"thread_start_us":102,"threads_started":1}
I20260812 03:03:15.409442 7105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:15.404420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:35433 (local address 127.5.96.129:58607)
0812 03:03:15.404877 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:15.404888 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:15.404964 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:15.405179 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:15.405182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:15.405201 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:15.405327 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:15.405333 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.406333 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.406336 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:15.407796 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:15.407807 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.408608 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.408612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:15.408738 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:15.408770 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 03:03:15.408848 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 03:03:15.408965 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":172,"thread_start_us":81,"threads_started":1}
I20260812 03:03:15.409915 6790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85e2137d9a3c4901be7acb75778099ac" candidate_uuid: "47a524539cc64aa0af4e5cf40461c9d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6aedbb883a914058a857dc57c0c36476" is_pre_election: true
I20260812 03:03:15.409999 6985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85e2137d9a3c4901be7acb75778099ac" candidate_uuid: "47a524539cc64aa0af4e5cf40461c9d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83d4e33f4c7d49f193473b07acd4399e" is_pre_election: true
I20260812 03:03:15.410322 6790 raft_consensus.cc:2454] T 85e2137d9a3c4901be7acb75778099ac P 6aedbb883a914058a857dc57c0c36476 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47a524539cc64aa0af4e5cf40461c9d5 in term 0.
I20260812 03:03:15.410807 6491 leader_election.cc:302] T 85e2137d9a3c4901be7acb75778099ac P 47a524539cc64aa0af4e5cf40461c9d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47a524539cc64aa0af4e5cf40461c9d5, 6aedbb883a914058a857dc57c0c36476; no voters:
I20260812 03:03:15.411036 7102 raft_consensus.cc:2781] T 85e2137d9a3c4901be7acb75778099ac P 47a524539cc64aa0af4e5cf40461c9d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:03:15.411114 7102 raft_consensus.cc:482] T 85e2137d9a3c4901be7acb75778099ac P 47a524539cc64aa0af4e5cf40461c9d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:03:15.411161 7102 raft_consensus.cc:3037] T 85e2137d9a3c4901be7acb75778099ac P 47a524539cc64aa0af4e5cf40461c9d5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:15.412225 7102 raft_consensus.cc:504] T 85e2137d9a3c4901be7acb75778099ac P 47a524539cc64aa0af4e5cf40461c9d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aedbb883a914058a857dc57c0c36476" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35967 } } peers { permanent_uuid: "83d4e33f4c7d49f193473b07acd4399e" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 35433 } } peers { permanent_uuid: "47a524539cc64aa0af4e5cf40461c9d5" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36137 } }
I20260812 03:03:15.412283 6985 raft_consensus.cc:2454] T 85e2137d9a3c4901be7acb75778099ac P 83d4e33f4c7d49f193473b07acd4399e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47a524539cc64aa0af4e5cf40461c9d5 in term 0.
I20260812 03:03:15.412540 7102 leader_election.cc:288] T 85e2137d9a3c4901be7acb75778099ac P 47a524539cc64aa0af4e5cf40461c9d5 [CANDIDATE]: Term 1 election: Requested vote from peers 6aedbb883a914058a857dc57c0c36476 (127.5.96.130:35967), 83d4e33f4c7d49f193473b07acd4399e (127.5.96.131:35433)
I20260812 03:03:15.412848 6790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85e2137d9a3c4901be7acb75778099ac" candidate_uuid: "47a524539cc64aa0af4e5cf40461c9d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6aedbb883a914058a857dc57c0c36476"
I20260812 03:03:15.412942 6790 raft_consensus.cc:3037] T 85e2137d9a3c4901be7acb75778099ac P 6aedbb883a914058a857dc57c0c36476 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:15.413038 6985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85e2137d9a3c4901be7acb75778099ac" candidate_uuid: "47a524539cc64aa0af4e5cf40461c9d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83d4e33f4c7d49f193473b07acd4399e"
I20260812 03:03:15.413148 6985 raft_consensus.cc:3037] T 85e2137d9a3c4901be7acb75778099ac P 83d4e33f4c7d49f193473b07acd4399e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:15.414084 6790 raft_consensus.cc:2454] T 85e2137d9a3c4901be7acb75778099ac P 6aedbb883a914058a857dc57c0c36476 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47a524539cc64aa0af4e5cf40461c9d5 in term 1.
I20260812 03:03:15.414211 6985 raft_consensus.cc:2454] T 85e2137d9a3c4901be7acb75778099ac P 83d4e33f4c7d49f193473b07acd4399e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47a524539cc64aa0af4e5cf40461c9d5 in term 1.
I20260812 03:03:15.414345 6491 leader_election.cc:302] T 85e2137d9a3c4901be7acb75778099ac P 47a524539cc64aa0af4e5cf40461c9d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47a524539cc64aa0af4e5cf40461c9d5, 6aedbb883a914058a857dc57c0c36476; no voters:
I20260812 03:03:15.414549 7102 raft_consensus.cc:2781] T 85e2137d9a3c4901be7acb75778099ac P 47a524539cc64aa0af4e5cf40461c9d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:15.414791 7102 raft_consensus.cc:686] T 85e2137d9a3c4901be7acb75778099ac P 47a524539cc64aa0af4e5cf40461c9d5 [term 1 LEADER]: Becoming Leader. State: Replica: 47a524539cc64aa0af4e5cf40461c9d5, State: Running, Role: LEADER
I20260812 03:03:15.415052 7102 consensus_queue.cc:237] T 85e2137d9a3c4901be7acb75778099ac P 47a524539cc64aa0af4e5cf40461c9d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aedbb883a914058a857dc57c0c36476" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35967 } } peers { permanent_uuid: "83d4e33f4c7d49f193473b07acd4399e" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 35433 } } peers { permanent_uuid: "47a524539cc64aa0af4e5cf40461c9d5" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36137 } }
I20260812 03:03:15.418231 6376 catalog_manager.cc:5495] T 85e2137d9a3c4901be7acb75778099ac P 47a524539cc64aa0af4e5cf40461c9d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 47a524539cc64aa0af4e5cf40461c9d5 (127.5.96.129). New cstate: current_term: 1 leader_uuid: "47a524539cc64aa0af4e5cf40461c9d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aedbb883a914058a857dc57c0c36476" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83d4e33f4c7d49f193473b07acd4399e" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47a524539cc64aa0af4e5cf40461c9d5" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36137 } health_report { overall_health: HEALTHY } } }
Aug 12 03:03:15 dist-test-slave-6bbx krb5kdc[6311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.129@KRBTEST.COM
I20260812 03:03:15.438777 7090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:15.430448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42480 (local address 127.5.96.129:36137)
0812 03:03:15.430567 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:15.430571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:15.430759 (+ 188us) server_negotiation.cc:407] Connection header received
0812 03:03:15.430875 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:15.430878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:15.430921 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:15.430982 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:15.431642 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.432155 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.432967 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.433180 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.433271 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:15.435505 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:15.435514 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:15.435528 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:15.435550 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:15.436934 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:15.437500 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:15.437506 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:15.437507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:15.437609 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:15.437885 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:15.437888 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:15.437889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:15.438122 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:15.438326 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:15.438511 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 03:03:15.438622 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 03:03:15 dist-test-slave-6bbx krb5kdc[6311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503795, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:03:15 dist-test-slave-6bbx krb5kdc[6311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503795, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.96.129@KRBTEST.COM
I20260812 03:03:15.465126 7090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:15.457610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42490 (local address 127.5.96.129:36137)
0812 03:03:15.457747 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:15.457751 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:15.457855 (+ 104us) server_negotiation.cc:407] Connection header received
0812 03:03:15.457971 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:15.457974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:15.458012 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:15.458074 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:15.458706 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.459231 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.460151 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.460370 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.460456 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:15.462527 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:15.462530 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:15.462532 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:15.462554 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:15.463600 (+ 1046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:15.463988 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:15.463992 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:15.463994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:15.464066 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:15.464312 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:15.464316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:15.464318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:15.464470 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:15.464712 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:15.464875 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 03:03:15.464982 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Aug 12 03:03:15 dist-test-slave-6bbx krb5kdc[6311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:03:15 dist-test-slave-6bbx krb5kdc[6311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.129@KRBTEST.COM
I20260812 03:03:15.488301 7090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:15.481192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42504 (local address 127.5.96.129:36137)
0812 03:03:15.481316 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:15.481326 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:15.481428 (+ 102us) server_negotiation.cc:407] Connection header received
0812 03:03:15.481518 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:15.481520 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:15.481553 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:15.481613 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:15.482329 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.482821 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.483605 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:15.483884 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:15.483964 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:15.485923 (+ 1959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:15.485926 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:15.485928 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:15.485949 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:15.487126 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:15.487423 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:15.487425 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:15.487426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:15.487489 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:15.487653 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:15.487655 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:15.487656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:15.487832 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:15.487942 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:15.488075 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 03:03:15.488155 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260812 03:03:15.489826 6528 tablet_service.cc:2908] Rejecting scan request for tablet 85e2137d9a3c4901be7acb75778099ac: Uninitialized: clean time has not yet been initialized
I20260812 03:03:15.492045 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 6465
I20260812 03:03:15.508003 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 6665
W20260812 03:03:15.512391 7067 tablet.cc:2369] T 85e2137d9a3c4901be7acb75778099ac P 83d4e33f4c7d49f193473b07acd4399e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:03:15.522331 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 6864
I20260812 03:03:15.536890 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 6331
2026-08-12T03:03:15Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2434 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:03:15 dist-test-slave-6bbx krb5kdc[7133](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:03:15 dist-test-slave-6bbx krb5kdc[7133](info): set up 2 sockets
Aug 12 03:03:15 dist-test-slave-6bbx krb5kdc[7133](info): commencing operation
krb5kdc: starting...
W20260812 03:03:16.682032 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:03:16 dist-test-slave-6bbx krb5kdc[7133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503796, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:03:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:03:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:03:16.789781 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:45861
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:42129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:45861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:16.897359 7155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:16.897601 7155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:16.897660 7155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:16.901547 7155 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:03:16.901595 7155 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 03:03:16.901619 7155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:16.901641 7155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:16.901670 7155 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:03:16.901700 7155 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:03:16.906788 7155 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:42129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:45861
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:45861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:16.907750 7155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:16.908664 7155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:16.916721 7164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:16.916770 7167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:16.916855 7155 server_base.cc:1034] running on GCE node
W20260812 03:03:16.916898 7162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:16.917557 7155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:16.918521 7155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:16.919697 7155 hybrid_clock.cc:648] HybridClock initialized: now 1786503796919676 us; error 30 us; skew 500 ppm
Aug 12 03:03:16 dist-test-slave-6bbx krb5kdc[7133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503796, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:16.926119 7155 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:03:16.926926 7155 webserver.cc:460] Webserver started at http://127.5.96.190:45173/ using document root <none> and password file <none>
I20260812 03:03:16.927431 7155 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:16.927491 7155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:16.927734 7155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:16.929664 7155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "59e293ad27564221b1ec6b76ebeb7de5"
format_stamp: "Formatted at 2026-08-12 03:03:16 on dist-test-slave-6bbx"
server_key: "0841f6605f5bfa881caad3446805b2f5"
server_key_iv: "5d94cdf8a74b35e5128304ceacfbf61e"
server_key_version: "encryptionkey@0"
I20260812 03:03:16.930199 7155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "59e293ad27564221b1ec6b76ebeb7de5"
format_stamp: "Formatted at 2026-08-12 03:03:16 on dist-test-slave-6bbx"
server_key: "0841f6605f5bfa881caad3446805b2f5"
server_key_iv: "5d94cdf8a74b35e5128304ceacfbf61e"
server_key_version: "encryptionkey@0"
I20260812 03:03:16.934028 7155 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260812 03:03:16.936239 7176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:16.937328 7155 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:16.937433 7155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "59e293ad27564221b1ec6b76ebeb7de5"
format_stamp: "Formatted at 2026-08-12 03:03:16 on dist-test-slave-6bbx"
server_key: "0841f6605f5bfa881caad3446805b2f5"
server_key_iv: "5d94cdf8a74b35e5128304ceacfbf61e"
server_key_version: "encryptionkey@0"
I20260812 03:03:16.937529 7155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:16.965242 7155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:16.968097 7155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:16.968259 7155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:16.975862 7155 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:45861
I20260812 03:03:16.975891 7274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:45861 every 8 connection(s)
I20260812 03:03:16.976706 7155 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:03:16.979285 7276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:16.984665 7276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59e293ad27564221b1ec6b76ebeb7de5: Bootstrap starting.
I20260812 03:03:16.984771 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 7155
I20260812 03:03:16.984879 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:03:16.985103 5506 external_mini_cluster.cc:1428] Setting key 226bdc4a7571d0a23680f96e422f98df
I20260812 03:03:16.987072 7276 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59e293ad27564221b1ec6b76ebeb7de5: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:16.987805 7276 log.cc:824] T 00000000000000000000000000000000 P 59e293ad27564221b1ec6b76ebeb7de5: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:16.989534 7276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59e293ad27564221b1ec6b76ebeb7de5: No bootstrap required, opened a new log
Aug 12 03:03:16 dist-test-slave-6bbx krb5kdc[7133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503796, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:16.993093 7276 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59e293ad27564221b1ec6b76ebeb7de5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59e293ad27564221b1ec6b76ebeb7de5" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45861 } }
I20260812 03:03:16.993335 7276 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59e293ad27564221b1ec6b76ebeb7de5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:16.993402 7276 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59e293ad27564221b1ec6b76ebeb7de5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59e293ad27564221b1ec6b76ebeb7de5, State: Initialized, Role: FOLLOWER
I20260812 03:03:16.993824 7276 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59e293ad27564221b1ec6b76ebeb7de5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59e293ad27564221b1ec6b76ebeb7de5" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45861 } }
I20260812 03:03:16.993961 7276 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59e293ad27564221b1ec6b76ebeb7de5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:03:16.994026 7276 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59e293ad27564221b1ec6b76ebeb7de5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:03:16.994131 7276 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59e293ad27564221b1ec6b76ebeb7de5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:16.994920 7276 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59e293ad27564221b1ec6b76ebeb7de5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59e293ad27564221b1ec6b76ebeb7de5" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45861 } }
I20260812 03:03:16.995239 7276 leader_election.cc:302] T 00000000000000000000000000000000 P 59e293ad27564221b1ec6b76ebeb7de5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59e293ad27564221b1ec6b76ebeb7de5; no voters:
I20260812 03:03:16.995524 7276 leader_election.cc:288] T 00000000000000000000000000000000 P 59e293ad27564221b1ec6b76ebeb7de5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:03:16.995625 7285 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59e293ad27564221b1ec6b76ebeb7de5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:16.995874 7285 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59e293ad27564221b1ec6b76ebeb7de5 [term 1 LEADER]: Becoming Leader. State: Replica: 59e293ad27564221b1ec6b76ebeb7de5, State: Running, Role: LEADER
I20260812 03:03:16.996186 7285 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59e293ad27564221b1ec6b76ebeb7de5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59e293ad27564221b1ec6b76ebeb7de5" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45861 } }
I20260812 03:03:16.996722 7276 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59e293ad27564221b1ec6b76ebeb7de5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:03:16.997735 7286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59e293ad27564221b1ec6b76ebeb7de5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59e293ad27564221b1ec6b76ebeb7de5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59e293ad27564221b1ec6b76ebeb7de5" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45861 } } }
I20260812 03:03:16.997864 7286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59e293ad27564221b1ec6b76ebeb7de5 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:16.997788 7287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59e293ad27564221b1ec6b76ebeb7de5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59e293ad27564221b1ec6b76ebeb7de5. Latest consensus state: current_term: 1 leader_uuid: "59e293ad27564221b1ec6b76ebeb7de5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59e293ad27564221b1ec6b76ebeb7de5" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45861 } } }
I20260812 03:03:16.998035 7287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59e293ad27564221b1ec6b76ebeb7de5 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:16.998240 7299 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:03:16.998693 7282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:16.986332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38612 (local address 127.5.96.190:45861)
0812 03:03:16.986742 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:16.986757 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:16.986791 (+ 34us) server_negotiation.cc:407] Connection header received
0812 03:03:16.987437 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:16.987455 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:16.987803 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:16.988110 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:16.988872 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:16.989868 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:16.990686 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:16.991130 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:16.991290 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:16.993594 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:16.993599 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:16.993612 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:16.993638 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:16.995274 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:16.995813 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:16.995820 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:16.995827 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:16.995975 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:16.996371 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:16.996375 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:16.996376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:16.996893 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:16.997082 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:16.997397 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 03:03:16.997708 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":143,"threads_started":1}
I20260812 03:03:17.001514 7299 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:03:17.006354 7299 catalog_manager.cc:1347] Generated new cluster ID: cf0a94ca08a149b190d7c59158fc09bf
I20260812 03:03:17.006420 7299 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:03:17.015223 7299 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:03:17.016152 7299 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:03:17.021137 7299 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59e293ad27564221b1ec6b76ebeb7de5: Generated new TSK 0
I20260812 03:03:17.021684 7299 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.96.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
I20260812 03:03:17.054200 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:0
--local_ip_for_outbound_sockets=127.5.96.129
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:45861
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:42129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:17.170199 7317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:17.170454 7317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:17.170513 7317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:17.174461 7317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:17.174520 7317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:17.174572 7317 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:03:17.174625 7317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:03:17.179786 7317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:42129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.96.190:45861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:17.180790 7317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:17.181818 7317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:17.191291 7329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:17.191308 7324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:17.191313 7325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:17.191365 7317 server_base.cc:1034] running on GCE node
I20260812 03:03:17.191920 7317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:17.192530 7317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:17.193684 7317 hybrid_clock.cc:648] HybridClock initialized: now 1786503797193667 us; error 27 us; skew 500 ppm
Aug 12 03:03:17 dist-test-slave-6bbx krb5kdc[7133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503797, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:17.196424 7317 init.cc:377] Logged in from keytab as kudu/127.5.96.129@KRBTEST.COM (short username kudu)
I20260812 03:03:17.197292 7317 webserver.cc:460] Webserver started at http://127.5.96.129:35079/ using document root <none> and password file <none>
I20260812 03:03:17.197839 7317 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:17.197906 7317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:17.198105 7317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:17.200168 7317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-0/data/instance:
uuid: "337faaa1672a48b0b2d658f4c5494cab"
format_stamp: "Formatted at 2026-08-12 03:03:17 on dist-test-slave-6bbx"
server_key: "5bf7566329399cf484260fd59da20250"
server_key_iv: "d774d93f3ca50d1d65383e00cb62c85c"
server_key_version: "encryptionkey@0"
I20260812 03:03:17.200696 7317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance:
uuid: "337faaa1672a48b0b2d658f4c5494cab"
format_stamp: "Formatted at 2026-08-12 03:03:17 on dist-test-slave-6bbx"
server_key: "5bf7566329399cf484260fd59da20250"
server_key_iv: "d774d93f3ca50d1d65383e00cb62c85c"
server_key_version: "encryptionkey@0"
I20260812 03:03:17.204208 7317 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:03:17.206569 7342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:17.207774 7317 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:03:17.207881 7317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "337faaa1672a48b0b2d658f4c5494cab"
format_stamp: "Formatted at 2026-08-12 03:03:17 on dist-test-slave-6bbx"
server_key: "5bf7566329399cf484260fd59da20250"
server_key_iv: "d774d93f3ca50d1d65383e00cb62c85c"
server_key_version: "encryptionkey@0"
I20260812 03:03:17.207973 7317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:17.228084 7317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:17.230710 7317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:17.230886 7317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:17.232350 7317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:17.232402 7317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:17.232453 7317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:17.232478 7317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:03:17 dist-test-slave-6bbx krb5kdc[7133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503797, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:17.242843 7317 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:33987
I20260812 03:03:17.242880 7511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:33987 every 8 connection(s)
I20260812 03:03:17.243835 7317 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:03:17.245649 7357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:17.233551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:45861 (local address 127.5.96.129:59311)
0812 03:03:17.234009 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:17.234050 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:17.234884 (+ 834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:17.235402 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:17.235411 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:17.235857 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:17.236385 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:17.236403 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.237368 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.237372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:17.238259 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:17.238269 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.238774 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.238779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:17.238926 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:17.239481 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:17.239504 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:17.241205 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:17.243068 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:17.243072 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:17.243083 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:17.243321 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:17.243601 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:17.243603 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:17.243605 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:17.243792 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:17.244223 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:17.244229 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:17.244501 (+ 272us) client_negotiation.cc:769] Sending connection context
0812 03:03:17.244683 (+ 182us) client_negotiation.cc:240] Negotiation successful
0812 03:03:17.244957 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":133,"threads_started":1}
I20260812 03:03:17.246243 7282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:17.233035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:59311 (local address 127.5.96.190:45861)
0812 03:03:17.233149 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:17.233153 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:17.234152 (+ 999us) server_negotiation.cc:407] Connection header received
0812 03:03:17.235083 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:17.235087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:17.235144 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:17.235236 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:17.236543 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.237255 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.238377 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.238619 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.238707 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:17.241390 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:17.241395 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:17.241398 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:17.241428 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:17.242970 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:17.243424 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:17.243428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:17.243430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:17.243520 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:17.243887 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:17.243890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:17.243892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:17.244117 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:17.244234 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:17.246020 (+ 1786us) server_negotiation.cc:299] Negotiation successful
0812 03:03:17.246134 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:03:17.246233 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 7317
I20260812 03:03:17.246332 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance
I20260812 03:03:17.246567 5506 external_mini_cluster.cc:1428] Setting key 71dd7c490313b6deae0c25ffb788287a
I20260812 03:03:17.247552 7513 heartbeater.cc:347] Connected to a master server at 127.5.96.190:45861
I20260812 03:03:17.247738 7513 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:17.248263 7513 heartbeater.cc:511] Master 127.5.96.190:45861 requested a full tablet report, sending...
I20260812 03:03:17.249814 7200 ts_manager.cc:194] Registered new tserver with Master: 337faaa1672a48b0b2d658f4c5494cab (127.5.96.129:33987)
I20260812 03:03:17.251142 7200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.129@KRBTEST.COM'} at 127.5.96.129:59311
I20260812 03:03:17.256449 7282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:17.249802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:45827 (local address 127.5.96.190:45861)
0812 03:03:17.249900 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:17.249904 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:17.250122 (+ 218us) server_negotiation.cc:407] Connection header received
0812 03:03:17.250207 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:17.250210 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:17.250254 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:17.250325 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:17.250799 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.251507 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.252486 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.252701 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.252786 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:17.253768 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:17.253772 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:17.253775 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:17.253797 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:17.255017 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:17.255386 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:17.255389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:17.255391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:17.255460 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:17.255712 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:17.255715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:17.255717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:17.255876 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:17.255960 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:17.256198 (+ 238us) server_negotiation.cc:299] Negotiation successful
0812 03:03:17.256310 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:03:17.256458 7522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:17.249728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:45861 (local address 127.5.96.129:45827)
0812 03:03:17.250043 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:17.250053 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:17.250139 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:17.250366 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:17.250369 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:17.250557 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:17.250676 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:17.250683 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.251760 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.251764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:17.252368 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:17.252378 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.252801 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.252804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:17.252892 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:17.253320 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:17.253334 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:17.253657 (+ 323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:17.255116 (+ 1459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:17.255119 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:17.255121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:17.255301 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:17.255539 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:17.255542 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:17.255544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:17.255622 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:17.255962 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:17.255966 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:17.256051 (+ 85us) client_negotiation.cc:769] Sending connection context
0812 03:03:17.256136 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 03:03:17.256268 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":91,"threads_started":1}
WARNING: no policy specified for kudu/127.5.96.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
I20260812 03:03:17.272186 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:0
--local_ip_for_outbound_sockets=127.5.96.130
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:45861
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:42129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:17.383560 7526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:17.383833 7526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:17.383888 7526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:17.387892 7526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:17.387953 7526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:17.388005 7526 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:03:17.388055 7526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:03:17.393064 7526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:42129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.96.190:45861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:17.394001 7526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:17.394948 7526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:03:17.404132 7526 server_base.cc:1034] running on GCE node
W20260812 03:03:17.404057 7536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:17.404049 7540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:17.404049 7534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:17.404662 7526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:17.405244 7526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:17.406414 7526 hybrid_clock.cc:648] HybridClock initialized: now 1786503797406398 us; error 28 us; skew 500 ppm
Aug 12 03:03:17 dist-test-slave-6bbx krb5kdc[7133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503797, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:17.409052 7526 init.cc:377] Logged in from keytab as kudu/127.5.96.130@KRBTEST.COM (short username kudu)
I20260812 03:03:17.409801 7526 webserver.cc:460] Webserver started at http://127.5.96.130:44101/ using document root <none> and password file <none>
I20260812 03:03:17.410277 7526 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:17.410332 7526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:17.410511 7526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:17.412480 7526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-1/data/instance:
uuid: "eb285e93c1b841d799d40d78773c908e"
format_stamp: "Formatted at 2026-08-12 03:03:17 on dist-test-slave-6bbx"
server_key: "468b6b9e294f80d965a9f70732538b67"
server_key_iv: "2bc554e859fe915537f35aafd78490a5"
server_key_version: "encryptionkey@0"
I20260812 03:03:17.412966 7526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance:
uuid: "eb285e93c1b841d799d40d78773c908e"
format_stamp: "Formatted at 2026-08-12 03:03:17 on dist-test-slave-6bbx"
server_key: "468b6b9e294f80d965a9f70732538b67"
server_key_iv: "2bc554e859fe915537f35aafd78490a5"
server_key_version: "encryptionkey@0"
I20260812 03:03:17.416453 7526 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:03:17.418761 7552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:17.419984 7526 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:03:17.420079 7526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "eb285e93c1b841d799d40d78773c908e"
format_stamp: "Formatted at 2026-08-12 03:03:17 on dist-test-slave-6bbx"
server_key: "468b6b9e294f80d965a9f70732538b67"
server_key_iv: "2bc554e859fe915537f35aafd78490a5"
server_key_version: "encryptionkey@0"
I20260812 03:03:17.420162 7526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:17.433717 7526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:17.436882 7526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:17.437038 7526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:17.438439 7526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:17.438493 7526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:17.438549 7526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:17.438580 7526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:03:17 dist-test-slave-6bbx krb5kdc[7133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503797, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:17.449129 7526 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:35107
I20260812 03:03:17.449190 7724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:35107 every 8 connection(s)
I20260812 03:03:17.450135 7526 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:03:17.451524 7282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:17.439209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:59839 (local address 127.5.96.190:45861)
0812 03:03:17.439361 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:17.439366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:17.440379 (+ 1013us) server_negotiation.cc:407] Connection header received
0812 03:03:17.441411 (+ 1032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:17.441415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:17.441480 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:17.441535 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:17.443183 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.444026 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.445220 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.445453 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.445544 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:17.447979 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:17.447984 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:17.447987 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:17.448017 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:17.449558 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:17.450005 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:17.450009 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:17.450010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:17.450105 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:17.450444 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:17.450448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:17.450450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:17.450678 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:17.450788 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:17.451296 (+ 508us) server_negotiation.cc:299] Negotiation successful
0812 03:03:17.451396 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:03:17.452282 7568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:17.439783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:45861 (local address 127.5.96.130:59839)
0812 03:03:17.440248 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:17.440290 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:17.441178 (+ 888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:17.441956 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:17.441965 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:17.442408 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:17.442992 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:17.443013 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.444159 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.444162 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:17.445103 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:17.445115 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.445604 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.445608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:17.445783 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:17.446280 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:17.446299 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:17.447811 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:17.449658 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:17.449663 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:17.449674 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:17.449906 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:17.450196 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:17.450199 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:17.450200 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:17.450348 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:17.450866 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:17.450872 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:17.451086 (+ 214us) client_negotiation.cc:769] Sending connection context
0812 03:03:17.451237 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 03:03:17.451472 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":123,"threads_started":1}
I20260812 03:03:17.453347 7727 heartbeater.cc:347] Connected to a master server at 127.5.96.190:45861
I20260812 03:03:17.453482 7727 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:17.453935 7727 heartbeater.cc:511] Master 127.5.96.190:45861 requested a full tablet report, sending...
I20260812 03:03:17.454882 7201 ts_manager.cc:194] Registered new tserver with Master: eb285e93c1b841d799d40d78773c908e (127.5.96.130:35107)
I20260812 03:03:17.455493 7201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.130@KRBTEST.COM'} at 127.5.96.130:59839
I20260812 03:03:17.459766 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 7526
I20260812 03:03:17.459872 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance
I20260812 03:03:17.460134 5506 external_mini_cluster.cc:1428] Setting key 6ca141b40365aaf34f83dd2d1879a14d
I20260812 03:03:17.461545 7282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:17.455389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:46185 (local address 127.5.96.190:45861)
0812 03:03:17.455506 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:17.455510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:17.455740 (+ 230us) server_negotiation.cc:407] Connection header received
0812 03:03:17.455887 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:17.455890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:17.455929 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:17.455987 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:17.456382 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.456860 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.457636 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.457845 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.457932 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:17.458831 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:17.458834 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:17.458837 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:17.458859 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:17.460037 (+ 1178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:17.460435 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:17.460439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:17.460440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:17.460528 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:17.460809 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:17.460811 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:17.460812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:17.461005 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:17.461080 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:17.461338 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 03:03:17.461443 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:03:17.461568 7733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:17.455318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:45861 (local address 127.5.96.130:46185)
0812 03:03:17.455643 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:17.455655 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:17.455806 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:17.456013 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:17.456016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:17.456185 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:17.456278 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:17.456284 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.456974 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.456977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:17.457531 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:17.457539 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.457945 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.457949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:17.458040 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:17.458424 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:17.458434 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:17.458699 (+ 265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:17.460146 (+ 1447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:17.460149 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:17.460152 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:17.460347 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:17.460629 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:17.460633 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:17.460635 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:17.460727 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:17.461091 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:17.461094 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:17.461182 (+ 88us) client_negotiation.cc:769] Sending connection context
0812 03:03:17.461280 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 03:03:17.461401 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":83,"threads_started":1}
WARNING: no policy specified for kudu/127.5.96.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
I20260812 03:03:17.486120 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:0
--local_ip_for_outbound_sockets=127.5.96.131
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:45861
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:42129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:17.597831 7738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:17.598079 7738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:17.598137 7738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:17.602056 7738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:17.602121 7738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:17.602173 7738 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:03:17.602224 7738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:03:17.607314 7738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:42129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.96.190:45861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:17.608318 7738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:17.609310 7738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:17.618606 7744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:17.618602 7746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:17.618773 7738 server_base.cc:1034] running on GCE node
W20260812 03:03:17.618618 7748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:17.619372 7738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:17.620011 7738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:17.621167 7738 hybrid_clock.cc:648] HybridClock initialized: now 1786503797621145 us; error 34 us; skew 500 ppm
Aug 12 03:03:17 dist-test-slave-6bbx krb5kdc[7133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503797, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:17.623926 7738 init.cc:377] Logged in from keytab as kudu/127.5.96.131@KRBTEST.COM (short username kudu)
I20260812 03:03:17.624773 7738 webserver.cc:460] Webserver started at http://127.5.96.131:40327/ using document root <none> and password file <none>
I20260812 03:03:17.625326 7738 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:17.625387 7738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:17.625577 7738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:17.627537 7738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-2/data/instance:
uuid: "1f5942cc327540c3810845302e4aa7ae"
format_stamp: "Formatted at 2026-08-12 03:03:17 on dist-test-slave-6bbx"
server_key: "a4cb6acf6e8d88bfcbd6e5ef409a8044"
server_key_iv: "454a44c08ef4f94ec80565c35ac44ad5"
server_key_version: "encryptionkey@0"
I20260812 03:03:17.628104 7738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance:
uuid: "1f5942cc327540c3810845302e4aa7ae"
format_stamp: "Formatted at 2026-08-12 03:03:17 on dist-test-slave-6bbx"
server_key: "a4cb6acf6e8d88bfcbd6e5ef409a8044"
server_key_iv: "454a44c08ef4f94ec80565c35ac44ad5"
server_key_version: "encryptionkey@0"
I20260812 03:03:17.631529 7738 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:03:17.633796 7759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:17.634927 7738 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:17.635031 7738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "1f5942cc327540c3810845302e4aa7ae"
format_stamp: "Formatted at 2026-08-12 03:03:17 on dist-test-slave-6bbx"
server_key: "a4cb6acf6e8d88bfcbd6e5ef409a8044"
server_key_iv: "454a44c08ef4f94ec80565c35ac44ad5"
server_key_version: "encryptionkey@0"
I20260812 03:03:17.635116 7738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:17.653604 7738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:17.656492 7738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:17.656656 7738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:17.657955 7738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:17.658003 7738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:17.658053 7738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:17.658079 7738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:03:17 dist-test-slave-6bbx krb5kdc[7133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503797, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:17.667961 7738 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:38573
I20260812 03:03:17.668012 7939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:38573 every 8 connection(s)
I20260812 03:03:17.668948 7738 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:03:17.670603 7282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:17.658684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:55199 (local address 127.5.96.190:45861)
0812 03:03:17.658829 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:17.658834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:17.659622 (+ 788us) server_negotiation.cc:407] Connection header received
0812 03:03:17.660600 (+ 978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:17.660604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:17.660661 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:17.660750 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:17.662207 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.662788 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.664063 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.664407 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.664529 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:17.667135 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:17.667138 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:17.667141 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:17.667170 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:17.668744 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:17.669242 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:17.669246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:17.669248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:17.669347 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:17.669717 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:17.669728 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:17.669730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:17.669931 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:17.670011 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:17.670391 (+ 380us) server_negotiation.cc:299] Negotiation successful
0812 03:03:17.670484 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:03:17.671365 7779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:17.659035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:45861 (local address 127.5.96.131:55199)
0812 03:03:17.659464 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:17.659513 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:17.660364 (+ 851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:17.661140 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:17.661149 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:17.661540 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:17.662040 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:17.662055 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.662908 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.662912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:17.663911 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:17.663923 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.664570 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.664574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:17.664798 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:17.665348 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:17.665363 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:17.666975 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:17.668855 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:17.668859 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:17.668869 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:17.669144 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:17.669459 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:17.669463 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:17.669465 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:17.669629 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:17.670012 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:17.670016 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:17.670210 (+ 194us) client_negotiation.cc:769] Sending connection context
0812 03:03:17.670353 (+ 143us) client_negotiation.cc:240] Negotiation successful
0812 03:03:17.670567 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":108,"threads_started":1}
I20260812 03:03:17.672416 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 7738
I20260812 03:03:17.672421 7941 heartbeater.cc:347] Connected to a master server at 127.5.96.190:45861
I20260812 03:03:17.672524 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance
I20260812 03:03:17.672554 7941 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:17.672796 5506 external_mini_cluster.cc:1428] Setting key 8ee140e544a7a295e1fccfc56ab0aa6e
I20260812 03:03:17.673019 7941 heartbeater.cc:511] Master 127.5.96.190:45861 requested a full tablet report, sending...
I20260812 03:03:17.674116 7200 ts_manager.cc:194] Registered new tserver with Master: 1f5942cc327540c3810845302e4aa7ae (127.5.96.131:38573)
I20260812 03:03:17.674870 7200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.131@KRBTEST.COM'} at 127.5.96.131:55199
I20260812 03:03:17.675007 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:03:17.680971 7282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:17.674642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:53117 (local address 127.5.96.190:45861)
0812 03:03:17.674745 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:17.674749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:17.674963 (+ 214us) server_negotiation.cc:407] Connection header received
0812 03:03:17.675137 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:17.675140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:17.675185 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:17.675266 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:17.675710 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.676175 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.676949 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.677161 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.677249 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:17.678136 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:17.678140 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:17.678143 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:17.678166 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:17.679443 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:17.679877 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:17.679880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:17.679882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:17.679968 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:17.680193 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:17.680197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:17.680198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:17.680431 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:17.680529 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:17.680738 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 03:03:17.680841 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 03:03:17.680981 7948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:17.674583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:45861 (local address 127.5.96.131:53117)
0812 03:03:17.674887 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:17.674901 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:17.675023 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:17.675295 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:17.675298 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:17.675490 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:17.675581 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:17.675587 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.676288 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.676291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:17.676845 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:17.676853 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.677254 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.677257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:17.677347 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:17.677732 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:17.677742 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:17.678017 (+ 275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:17.679543 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:17.679545 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:17.679546 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:17.679790 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:17.680044 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:17.680046 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:17.680047 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:17.680114 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:17.680511 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:17.680513 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:17.680579 (+ 66us) client_negotiation.cc:769] Sending connection context
0812 03:03:17.680657 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 03:03:17.680797 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":96,"threads_started":1}
Aug 12 03:03:17 dist-test-slave-6bbx krb5kdc[7133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503797, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:03:17 dist-test-slave-6bbx krb5kdc[7133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503797, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:17.701620 7282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:17.694829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55326 (local address 127.5.96.190:45861)
0812 03:03:17.694922 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:17.694925 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:17.694936 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:03:17.694963 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:17.694965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:17.694997 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:17.695058 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:17.695629 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.696135 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.696904 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.697103 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.697167 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:17.699185 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:17.699188 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:17.699189 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:17.699210 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:17.700444 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:17.700776 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:17.700778 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:17.700780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:17.700842 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:17.701055 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:17.701057 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:17.701058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:17.701193 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:17.701295 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:17.701437 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 03:03:17.701520 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 03:03:17.704139 7200 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55326:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:03:17.706951 7200 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:03:17.720122 7966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:17.714486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:33987 (local address 127.0.0.1:57754)
0812 03:03:17.715124 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:17.715167 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:17.715294 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:17.716759 (+ 1465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:17.716762 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:17.716781 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:17.716870 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:17.716877 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.717770 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.717773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:17.718860 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:17.718870 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.719587 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.719591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:17.719734 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:17.719777 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 03:03:17.719876 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 03:03:17.719989 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"spinlock_wait_cycles":33024,"thread_start_us":71,"threads_started":1}
I20260812 03:03:17.720132 7965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:17.714600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57754 (local address 127.5.96.129:33987)
0812 03:03:17.715559 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:17.715573 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:17.715592 (+ 19us) server_negotiation.cc:407] Connection header received
0812 03:03:17.715650 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:17.715655 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:17.715882 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:17.716777 (+ 895us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:17.716974 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.717666 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.718969 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.719479 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.719573 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:17.719762 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:17.719913 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 03:03:17.719998 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":876,"thread_start_us":54,"threads_started":1}
I20260812 03:03:17.720585 7964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:17.714639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:38573 (local address 127.0.0.1:46738)
0812 03:03:17.715121 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:17.715166 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:17.715293 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:17.716584 (+ 1291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:17.716591 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:17.716627 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:17.716824 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:17.716830 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.718218 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.718221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:17.719128 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:17.719138 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.720173 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.720176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:17.720307 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:17.720328 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 03:03:17.720392 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 03:03:17.720455 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"mutex_wait_us":75,"spinlock_wait_cycles":7680,"thread_start_us":103,"threads_started":1}
I20260812 03:03:17.720674 7967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:17.714888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46738 (local address 127.5.96.131:38573)
0812 03:03:17.716150 (+ 1262us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:17.716162 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:17.716182 (+ 20us) server_negotiation.cc:407] Connection header received
0812 03:03:17.716252 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:17.716257 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:17.716426 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:17.716584 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:17.716951 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.718093 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.719241 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.720042 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.720287 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:17.720438 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:17.720484 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 03:03:17.720557 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1172,"thread_start_us":101,"threads_started":1}
I20260812 03:03:17.722535 7971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:17.715591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:35107 (local address 127.0.0.1:56752)
0812 03:03:17.715818 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:17.715829 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:17.715906 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:17.716584 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:17.716591 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:17.716625 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:17.716817 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:17.716826 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.718216 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.718233 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:17.720835 (+ 2602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:17.720846 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.722157 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.722160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:17.722246 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:17.722268 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:03:17.722344 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 03:03:17.722408 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":133,"spinlock_wait_cycles":4608,"thread_start_us":116,"threads_started":1}
I20260812 03:03:17.722636 7973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:17.715799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56752 (local address 127.5.96.130:35107)
0812 03:03:17.716206 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:17.716216 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:17.716233 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:03:17.716308 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:17.716312 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:17.716452 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:17.716584 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:17.716954 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.718092 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.720958 (+ 2866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:17.721828 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:17.721934 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:17.722076 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:17.722401 (+ 325us) server_negotiation.cc:299] Negotiation successful
0812 03:03:17.722498 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":123,"threads_started":1}
I20260812 03:03:17.723254 7421 tablet_service.cc:1467] Processing CreateTablet for tablet 04d2cb0b416e47069e00bfe4d7aa47e1 (DEFAULT_TABLE table=test-table [id=b50fa069035042b4a367df112d8234d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:17.723244 7841 tablet_service.cc:1467] Processing CreateTablet for tablet 04d2cb0b416e47069e00bfe4d7aa47e1 (DEFAULT_TABLE table=test-table [id=b50fa069035042b4a367df112d8234d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:17.724148 7421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04d2cb0b416e47069e00bfe4d7aa47e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:17.724165 7841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04d2cb0b416e47069e00bfe4d7aa47e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:17.731950 7976 tablet_bootstrap.cc:492] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 1f5942cc327540c3810845302e4aa7ae: Bootstrap starting.
I20260812 03:03:17.732425 7634 tablet_service.cc:1467] Processing CreateTablet for tablet 04d2cb0b416e47069e00bfe4d7aa47e1 (DEFAULT_TABLE table=test-table [id=b50fa069035042b4a367df112d8234d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:17.732918 7977 tablet_bootstrap.cc:492] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab: Bootstrap starting.
I20260812 03:03:17.733402 7634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04d2cb0b416e47069e00bfe4d7aa47e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:17.734010 7976 tablet_bootstrap.cc:654] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 1f5942cc327540c3810845302e4aa7ae: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:17.734871 7977 tablet_bootstrap.cc:654] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:17.734836 7976 log.cc:824] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 1f5942cc327540c3810845302e4aa7ae: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:17.735658 7977 log.cc:824] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:17.736956 7976 tablet_bootstrap.cc:492] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 1f5942cc327540c3810845302e4aa7ae: No bootstrap required, opened a new log
I20260812 03:03:17.737146 7976 ts_tablet_manager.cc:1397] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 1f5942cc327540c3810845302e4aa7ae: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 03:03:17.737749 7977 tablet_bootstrap.cc:492] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab: No bootstrap required, opened a new log
I20260812 03:03:17.737942 7977 ts_tablet_manager.cc:1397] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 03:03:17.740247 7980 tablet_bootstrap.cc:492] T 04d2cb0b416e47069e00bfe4d7aa47e1 P eb285e93c1b841d799d40d78773c908e: Bootstrap starting.
I20260812 03:03:17.740166 7976 raft_consensus.cc:348] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 1f5942cc327540c3810845302e4aa7ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } }
I20260812 03:03:17.740564 7976 raft_consensus.cc:374] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 1f5942cc327540c3810845302e4aa7ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:17.740765 7976 raft_consensus.cc:729] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 1f5942cc327540c3810845302e4aa7ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f5942cc327540c3810845302e4aa7ae, State: Initialized, Role: FOLLOWER
I20260812 03:03:17.741288 7976 consensus_queue.cc:260] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 1f5942cc327540c3810845302e4aa7ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } }
I20260812 03:03:17.741729 7977 raft_consensus.cc:348] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } }
I20260812 03:03:17.741972 7941 heartbeater.cc:503] Master 127.5.96.190:45861 was elected leader, sending a full tablet report...
I20260812 03:03:17.742015 7977 raft_consensus.cc:374] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:17.742254 7977 raft_consensus.cc:729] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337faaa1672a48b0b2d658f4c5494cab, State: Initialized, Role: FOLLOWER
I20260812 03:03:17.742341 7980 tablet_bootstrap.cc:654] T 04d2cb0b416e47069e00bfe4d7aa47e1 P eb285e93c1b841d799d40d78773c908e: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:17.742682 7977 consensus_queue.cc:260] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } }
I20260812 03:03:17.743163 7980 log.cc:824] T 04d2cb0b416e47069e00bfe4d7aa47e1 P eb285e93c1b841d799d40d78773c908e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:17.743484 7976 ts_tablet_manager.cc:1428] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 1f5942cc327540c3810845302e4aa7ae: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 03:03:17.743628 7977 ts_tablet_manager.cc:1428] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 03:03:17.744050 7513 heartbeater.cc:503] Master 127.5.96.190:45861 was elected leader, sending a full tablet report...
W20260812 03:03:17.745046 7515 tablet.cc:2369] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:03:17.745098 7980 tablet_bootstrap.cc:492] T 04d2cb0b416e47069e00bfe4d7aa47e1 P eb285e93c1b841d799d40d78773c908e: No bootstrap required, opened a new log
I20260812 03:03:17.745263 7980 ts_tablet_manager.cc:1397] T 04d2cb0b416e47069e00bfe4d7aa47e1 P eb285e93c1b841d799d40d78773c908e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 03:03:17.748626 7980 raft_consensus.cc:348] T 04d2cb0b416e47069e00bfe4d7aa47e1 P eb285e93c1b841d799d40d78773c908e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } }
I20260812 03:03:17.748834 7980 raft_consensus.cc:374] T 04d2cb0b416e47069e00bfe4d7aa47e1 P eb285e93c1b841d799d40d78773c908e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:17.748870 7980 raft_consensus.cc:729] T 04d2cb0b416e47069e00bfe4d7aa47e1 P eb285e93c1b841d799d40d78773c908e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb285e93c1b841d799d40d78773c908e, State: Initialized, Role: FOLLOWER
I20260812 03:03:17.749255 7980 consensus_queue.cc:260] T 04d2cb0b416e47069e00bfe4d7aa47e1 P eb285e93c1b841d799d40d78773c908e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } }
I20260812 03:03:17.749814 7727 heartbeater.cc:503] Master 127.5.96.190:45861 was elected leader, sending a full tablet report...
I20260812 03:03:17.750058 7980 ts_tablet_manager.cc:1428] T 04d2cb0b416e47069e00bfe4d7aa47e1 P eb285e93c1b841d799d40d78773c908e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260812 03:03:17.919886 7942 tablet.cc:2369] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 1f5942cc327540c3810845302e4aa7ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:03:17.951242 7728 tablet.cc:2369] T 04d2cb0b416e47069e00bfe4d7aa47e1 P eb285e93c1b841d799d40d78773c908e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:03:18.062762 7986 raft_consensus.cc:482] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:03:18.062976 7986 raft_consensus.cc:504] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } }
I20260812 03:03:18.064180 7986 leader_election.cc:288] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f5942cc327540c3810845302e4aa7ae (127.5.96.131:38573), eb285e93c1b841d799d40d78773c908e (127.5.96.130:35107)
I20260812 03:03:18.067951 7967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:18.064573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:52749 (local address 127.5.96.131:38573)
0812 03:03:18.064769 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:18.064773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:18.064808 (+ 35us) server_negotiation.cc:407] Connection header received
0812 03:03:18.064963 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:18.064966 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:18.065007 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:18.065071 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:18.065357 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.065874 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.066958 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.067349 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.067461 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:18.067515 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:18.067748 (+ 233us) server_negotiation.cc:299] Negotiation successful
0812 03:03:18.067818 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260812 03:03:18.067961 7973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:18.064552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:56679 (local address 127.5.96.130:35107)
0812 03:03:18.064739 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:18.064744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:18.064938 (+ 194us) server_negotiation.cc:407] Connection header received
0812 03:03:18.065060 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:18.065063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:18.065130 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:18.065195 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:18.065449 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.065938 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.066962 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.067356 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.067463 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:18.067516 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:18.067767 (+ 251us) server_negotiation.cc:299] Negotiation successful
0812 03:03:18.067836 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260812 03:03:18.067976 7990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:18.064396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:38573 (local address 127.5.96.129:52749)
0812 03:03:18.064733 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:18.064748 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:18.064858 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:18.065118 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:18.065121 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:18.065139 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:18.065248 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:18.065253 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.065984 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.065988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:18.066829 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:18.066838 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.067461 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.067464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:18.067607 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:18.067638 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 03:03:18.067730 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 03:03:18.067831 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":51,"thread_start_us":78,"threads_started":1}
I20260812 03:03:18.067982 7989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:18.064444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:35107 (local address 127.5.96.129:56679)
0812 03:03:18.064853 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:18.064864 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:18.064961 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:18.065223 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:18.065225 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:18.065242 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:18.065348 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:18.065353 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.066047 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.066051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:18.066840 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:18.066849 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.067467 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.067470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:18.067600 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:18.067650 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 03:03:18.067738 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 03:03:18.067839 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":81,"thread_start_us":77,"threads_started":1}
I20260812 03:03:18.068619 7875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04d2cb0b416e47069e00bfe4d7aa47e1" candidate_uuid: "337faaa1672a48b0b2d658f4c5494cab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f5942cc327540c3810845302e4aa7ae" is_pre_election: true
I20260812 03:03:18.068701 7662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04d2cb0b416e47069e00bfe4d7aa47e1" candidate_uuid: "337faaa1672a48b0b2d658f4c5494cab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb285e93c1b841d799d40d78773c908e" is_pre_election: true
I20260812 03:03:18.068933 7875 raft_consensus.cc:2454] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 1f5942cc327540c3810845302e4aa7ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337faaa1672a48b0b2d658f4c5494cab in term 0.
I20260812 03:03:18.068933 7662 raft_consensus.cc:2454] T 04d2cb0b416e47069e00bfe4d7aa47e1 P eb285e93c1b841d799d40d78773c908e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337faaa1672a48b0b2d658f4c5494cab in term 0.
I20260812 03:03:18.069398 7344 leader_election.cc:302] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337faaa1672a48b0b2d658f4c5494cab, eb285e93c1b841d799d40d78773c908e; no voters:
I20260812 03:03:18.069674 7986 raft_consensus.cc:2781] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:03:18.069777 7986 raft_consensus.cc:482] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:03:18.069834 7986 raft_consensus.cc:3037] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:18.070838 7986 raft_consensus.cc:504] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } }
I20260812 03:03:18.071173 7986 leader_election.cc:288] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab [CANDIDATE]: Term 1 election: Requested vote from peers 1f5942cc327540c3810845302e4aa7ae (127.5.96.131:38573), eb285e93c1b841d799d40d78773c908e (127.5.96.130:35107)
I20260812 03:03:18.071499 7875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04d2cb0b416e47069e00bfe4d7aa47e1" candidate_uuid: "337faaa1672a48b0b2d658f4c5494cab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f5942cc327540c3810845302e4aa7ae"
I20260812 03:03:18.071527 7662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04d2cb0b416e47069e00bfe4d7aa47e1" candidate_uuid: "337faaa1672a48b0b2d658f4c5494cab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb285e93c1b841d799d40d78773c908e"
I20260812 03:03:18.071597 7875 raft_consensus.cc:3037] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 1f5942cc327540c3810845302e4aa7ae [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:18.071655 7662 raft_consensus.cc:3037] T 04d2cb0b416e47069e00bfe4d7aa47e1 P eb285e93c1b841d799d40d78773c908e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:18.072777 7662 raft_consensus.cc:2454] T 04d2cb0b416e47069e00bfe4d7aa47e1 P eb285e93c1b841d799d40d78773c908e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337faaa1672a48b0b2d658f4c5494cab in term 1.
I20260812 03:03:18.073069 7344 leader_election.cc:302] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337faaa1672a48b0b2d658f4c5494cab, eb285e93c1b841d799d40d78773c908e; no voters:
I20260812 03:03:18.073174 7875 raft_consensus.cc:2454] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 1f5942cc327540c3810845302e4aa7ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337faaa1672a48b0b2d658f4c5494cab in term 1.
I20260812 03:03:18.073285 7986 raft_consensus.cc:2781] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:18.073457 7986 raft_consensus.cc:686] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab [term 1 LEADER]: Becoming Leader. State: Replica: 337faaa1672a48b0b2d658f4c5494cab, State: Running, Role: LEADER
I20260812 03:03:18.073764 7986 consensus_queue.cc:237] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } }
I20260812 03:03:18.076700 7200 catalog_manager.cc:5495] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab reported cstate change: term changed from 0 to 1, leader changed from <none> to 337faaa1672a48b0b2d658f4c5494cab (127.5.96.129). New cstate: current_term: 1 leader_uuid: "337faaa1672a48b0b2d658f4c5494cab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } health_report { overall_health: UNKNOWN } } }
I20260812 03:03:18.172982 7965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:18.169215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57766 (local address 127.5.96.129:33987)
0812 03:03:18.169355 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:18.169360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:18.169373 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:03:18.169575 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:18.169578 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:18.169638 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:18.169731 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:03:18.169987 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.170554 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.171434 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.171653 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.171767 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:18.172109 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:03:18.172583 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:03:18.172689 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:18.172792 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 03:03:18.172846 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 03:03:18.179066 7662 raft_consensus.cc:1261] T 04d2cb0b416e47069e00bfe4d7aa47e1 P eb285e93c1b841d799d40d78773c908e [term 1 FOLLOWER]: Refusing update from remote peer 337faaa1672a48b0b2d658f4c5494cab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:18.179798 7986 consensus_queue.cc:1035] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:18.180512 7875 raft_consensus.cc:1261] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 1f5942cc327540c3810845302e4aa7ae [term 1 FOLLOWER]: Refusing update from remote peer 337faaa1672a48b0b2d658f4c5494cab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:18.181047 7986 consensus_queue.cc:1035] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:18.187292 7995 mvcc.cc:205] Tried to move back new op lower bound from 7317519557333385216 to 7317519556912427008. Current Snapshot: MvccSnapshot[applied={T|T < 7317519557333385216}]
I20260812 03:03:18.187317 7996 mvcc.cc:205] Tried to move back new op lower bound from 7317519557333385216 to 7317519556912427008. Current Snapshot: MvccSnapshot[applied={T|T < 7317519557333385216}]
I20260812 03:03:18.198737 7201 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:18.198957 7201 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:18.201362 7201 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 59e293ad27564221b1ec6b76ebeb7de5: Sending DeleteTablet for 3 replicas of tablet 04d2cb0b416e47069e00bfe4d7aa47e1
I20260812 03:03:18.202296 7421 tablet_service.cc:1514] Processing DeleteTablet for tablet 04d2cb0b416e47069e00bfe4d7aa47e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:18 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:57754
I20260812 03:03:18.202522 7841 tablet_service.cc:1514] Processing DeleteTablet for tablet 04d2cb0b416e47069e00bfe4d7aa47e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:18 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:46738
I20260812 03:03:18.202533 7634 tablet_service.cc:1514] Processing DeleteTablet for tablet 04d2cb0b416e47069e00bfe4d7aa47e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:18 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:56752
I20260812 03:03:18.202935 8015 tablet_replica.cc:330] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 1f5942cc327540c3810845302e4aa7ae: stopping tablet replica
I20260812 03:03:18.202962 8017 tablet_replica.cc:330] T 04d2cb0b416e47069e00bfe4d7aa47e1 P eb285e93c1b841d799d40d78773c908e: stopping tablet replica
I20260812 03:03:18.203238 8017 raft_consensus.cc:2229] T 04d2cb0b416e47069e00bfe4d7aa47e1 P eb285e93c1b841d799d40d78773c908e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:18.203289 8015 raft_consensus.cc:2229] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 1f5942cc327540c3810845302e4aa7ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:18.203608 8017 raft_consensus.cc:2258] T 04d2cb0b416e47069e00bfe4d7aa47e1 P eb285e93c1b841d799d40d78773c908e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:18.203619 8015 raft_consensus.cc:2258] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 1f5942cc327540c3810845302e4aa7ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:18.203806 7201 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55326:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:03:18.204195 7201 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:03:18.205089 8014 tablet_replica.cc:330] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab: stopping tablet replica
I20260812 03:03:18.205276 8015 ts_tablet_manager.cc:1905] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 1f5942cc327540c3810845302e4aa7ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:18.205370 8014 raft_consensus.cc:2229] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:03:18.205772 8014 raft_consensus.cc:2258] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:18.207036 8014 ts_tablet_manager.cc:1905] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:18.208107 8017 ts_tablet_manager.cc:1905] T 04d2cb0b416e47069e00bfe4d7aa47e1 P eb285e93c1b841d799d40d78773c908e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:18.209908 8014 ts_tablet_manager.cc:1918] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:03:18.210063 8014 log.cc:1196] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-0/wal/wals/04d2cb0b416e47069e00bfe4d7aa47e1
I20260812 03:03:18.210319 7634 tablet_service.cc:1467] Processing CreateTablet for tablet aac7e3cdf7d14490bbdde38b4b38b057 (DEFAULT_TABLE table=test-table [id=5be4e835233f422f8943482c519597d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:18.210443 8014 ts_tablet_manager.cc:1939] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 337faaa1672a48b0b2d658f4c5494cab: Deleting consensus metadata
I20260812 03:03:18.210649 7634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aac7e3cdf7d14490bbdde38b4b38b057. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:18.210659 8017 ts_tablet_manager.cc:1918] T 04d2cb0b416e47069e00bfe4d7aa47e1 P eb285e93c1b841d799d40d78773c908e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:03:18.210809 8017 log.cc:1196] T 04d2cb0b416e47069e00bfe4d7aa47e1 P eb285e93c1b841d799d40d78773c908e: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-1/wal/wals/04d2cb0b416e47069e00bfe4d7aa47e1
I20260812 03:03:18.211082 8017 ts_tablet_manager.cc:1939] T 04d2cb0b416e47069e00bfe4d7aa47e1 P eb285e93c1b841d799d40d78773c908e: Deleting consensus metadata
I20260812 03:03:18.211948 7841 tablet_service.cc:1467] Processing CreateTablet for tablet aac7e3cdf7d14490bbdde38b4b38b057 (DEFAULT_TABLE table=test-table [id=5be4e835233f422f8943482c519597d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:18.212206 7841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aac7e3cdf7d14490bbdde38b4b38b057. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:18.212811 7980 tablet_bootstrap.cc:492] T aac7e3cdf7d14490bbdde38b4b38b057 P eb285e93c1b841d799d40d78773c908e: Bootstrap starting.
I20260812 03:03:18.213672 7980 tablet_bootstrap.cc:654] T aac7e3cdf7d14490bbdde38b4b38b057 P eb285e93c1b841d799d40d78773c908e: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:18.214092 7976 tablet_bootstrap.cc:492] T aac7e3cdf7d14490bbdde38b4b38b057 P 1f5942cc327540c3810845302e4aa7ae: Bootstrap starting.
I20260812 03:03:18.214359 7179 catalog_manager.cc:4841] TS eb285e93c1b841d799d40d78773c908e (127.5.96.130:35107): tablet 04d2cb0b416e47069e00bfe4d7aa47e1 (table test-table [id=b50fa069035042b4a367df112d8234d0]) successfully deleted
I20260812 03:03:18.214517 7179 catalog_manager.cc:4841] TS 337faaa1672a48b0b2d658f4c5494cab (127.5.96.129:33987): tablet 04d2cb0b416e47069e00bfe4d7aa47e1 (table test-table [id=b50fa069035042b4a367df112d8234d0]) successfully deleted
I20260812 03:03:18.210029 7421 tablet_service.cc:1467] Processing CreateTablet for tablet aac7e3cdf7d14490bbdde38b4b38b057 (DEFAULT_TABLE table=test-table [id=5be4e835233f422f8943482c519597d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:18.214929 7976 tablet_bootstrap.cc:654] T aac7e3cdf7d14490bbdde38b4b38b057 P 1f5942cc327540c3810845302e4aa7ae: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:18.214982 7421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aac7e3cdf7d14490bbdde38b4b38b057. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:18.215176 7980 tablet_bootstrap.cc:492] T aac7e3cdf7d14490bbdde38b4b38b057 P eb285e93c1b841d799d40d78773c908e: No bootstrap required, opened a new log
I20260812 03:03:18.215251 7980 ts_tablet_manager.cc:1397] T aac7e3cdf7d14490bbdde38b4b38b057 P eb285e93c1b841d799d40d78773c908e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:18.215687 7980 raft_consensus.cc:348] T aac7e3cdf7d14490bbdde38b4b38b057 P eb285e93c1b841d799d40d78773c908e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } }
I20260812 03:03:18.215793 7980 raft_consensus.cc:374] T aac7e3cdf7d14490bbdde38b4b38b057 P eb285e93c1b841d799d40d78773c908e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:18.215822 7980 raft_consensus.cc:729] T aac7e3cdf7d14490bbdde38b4b38b057 P eb285e93c1b841d799d40d78773c908e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb285e93c1b841d799d40d78773c908e, State: Initialized, Role: FOLLOWER
I20260812 03:03:18.215941 7980 consensus_queue.cc:260] T aac7e3cdf7d14490bbdde38b4b38b057 P eb285e93c1b841d799d40d78773c908e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } }
I20260812 03:03:18.216110 7976 tablet_bootstrap.cc:492] T aac7e3cdf7d14490bbdde38b4b38b057 P 1f5942cc327540c3810845302e4aa7ae: No bootstrap required, opened a new log
I20260812 03:03:18.216157 7980 ts_tablet_manager.cc:1428] T aac7e3cdf7d14490bbdde38b4b38b057 P eb285e93c1b841d799d40d78773c908e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:03:18.216197 7976 ts_tablet_manager.cc:1397] T aac7e3cdf7d14490bbdde38b4b38b057 P 1f5942cc327540c3810845302e4aa7ae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:03:18.216616 7976 raft_consensus.cc:348] T aac7e3cdf7d14490bbdde38b4b38b057 P 1f5942cc327540c3810845302e4aa7ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } }
I20260812 03:03:18.216722 7976 raft_consensus.cc:374] T aac7e3cdf7d14490bbdde38b4b38b057 P 1f5942cc327540c3810845302e4aa7ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:18.216750 7976 raft_consensus.cc:729] T aac7e3cdf7d14490bbdde38b4b38b057 P 1f5942cc327540c3810845302e4aa7ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f5942cc327540c3810845302e4aa7ae, State: Initialized, Role: FOLLOWER
I20260812 03:03:18.216881 7976 consensus_queue.cc:260] T aac7e3cdf7d14490bbdde38b4b38b057 P 1f5942cc327540c3810845302e4aa7ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } }
I20260812 03:03:18.217136 7976 ts_tablet_manager.cc:1428] T aac7e3cdf7d14490bbdde38b4b38b057 P 1f5942cc327540c3810845302e4aa7ae: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:03:18.217478 7977 tablet_bootstrap.cc:492] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab: Bootstrap starting.
I20260812 03:03:18.218364 7977 tablet_bootstrap.cc:654] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:18.219136 8015 ts_tablet_manager.cc:1918] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 1f5942cc327540c3810845302e4aa7ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:03:18.219216 8015 log.cc:1196] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 1f5942cc327540c3810845302e4aa7ae: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-2/wal/wals/04d2cb0b416e47069e00bfe4d7aa47e1
I20260812 03:03:18.219467 7977 tablet_bootstrap.cc:492] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab: No bootstrap required, opened a new log
I20260812 03:03:18.219494 8015 ts_tablet_manager.cc:1939] T 04d2cb0b416e47069e00bfe4d7aa47e1 P 1f5942cc327540c3810845302e4aa7ae: Deleting consensus metadata
I20260812 03:03:18.219535 7977 ts_tablet_manager.cc:1397] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:03:18.219991 7977 raft_consensus.cc:348] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } }
I20260812 03:03:18.220085 7977 raft_consensus.cc:374] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:18.220113 7977 raft_consensus.cc:729] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337faaa1672a48b0b2d658f4c5494cab, State: Initialized, Role: FOLLOWER
I20260812 03:03:18.220220 7977 consensus_queue.cc:260] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } }
I20260812 03:03:18.220445 7977 ts_tablet_manager.cc:1428] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:03:18.221601 7180 catalog_manager.cc:4841] TS 1f5942cc327540c3810845302e4aa7ae (127.5.96.131:38573): tablet 04d2cb0b416e47069e00bfe4d7aa47e1 (table test-table [id=b50fa069035042b4a367df112d8234d0]) successfully deleted
I20260812 03:03:18.268487 8002 raft_consensus.cc:482] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:03:18.268601 8002 raft_consensus.cc:504] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } }
I20260812 03:03:18.269018 8002 leader_election.cc:288] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb285e93c1b841d799d40d78773c908e (127.5.96.130:35107), 1f5942cc327540c3810845302e4aa7ae (127.5.96.131:38573)
I20260812 03:03:18.269361 7875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac7e3cdf7d14490bbdde38b4b38b057" candidate_uuid: "337faaa1672a48b0b2d658f4c5494cab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f5942cc327540c3810845302e4aa7ae" is_pre_election: true
I20260812 03:03:18.269371 7662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac7e3cdf7d14490bbdde38b4b38b057" candidate_uuid: "337faaa1672a48b0b2d658f4c5494cab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb285e93c1b841d799d40d78773c908e" is_pre_election: true
I20260812 03:03:18.269495 7875 raft_consensus.cc:2454] T aac7e3cdf7d14490bbdde38b4b38b057 P 1f5942cc327540c3810845302e4aa7ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337faaa1672a48b0b2d658f4c5494cab in term 0.
I20260812 03:03:18.269498 7662 raft_consensus.cc:2454] T aac7e3cdf7d14490bbdde38b4b38b057 P eb285e93c1b841d799d40d78773c908e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337faaa1672a48b0b2d658f4c5494cab in term 0.
I20260812 03:03:18.269796 7345 leader_election.cc:302] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f5942cc327540c3810845302e4aa7ae, 337faaa1672a48b0b2d658f4c5494cab; no voters:
I20260812 03:03:18.269968 8002 raft_consensus.cc:2781] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:03:18.270026 8002 raft_consensus.cc:482] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:03:18.270069 8002 raft_consensus.cc:3037] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:18.270951 8002 raft_consensus.cc:504] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } }
I20260812 03:03:18.271270 8002 leader_election.cc:288] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab [CANDIDATE]: Term 1 election: Requested vote from peers eb285e93c1b841d799d40d78773c908e (127.5.96.130:35107), 1f5942cc327540c3810845302e4aa7ae (127.5.96.131:38573)
I20260812 03:03:18.271625 7875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac7e3cdf7d14490bbdde38b4b38b057" candidate_uuid: "337faaa1672a48b0b2d658f4c5494cab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f5942cc327540c3810845302e4aa7ae"
I20260812 03:03:18.271772 7875 raft_consensus.cc:3037] T aac7e3cdf7d14490bbdde38b4b38b057 P 1f5942cc327540c3810845302e4aa7ae [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:18.272071 7662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac7e3cdf7d14490bbdde38b4b38b057" candidate_uuid: "337faaa1672a48b0b2d658f4c5494cab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb285e93c1b841d799d40d78773c908e"
I20260812 03:03:18.272188 7662 raft_consensus.cc:3037] T aac7e3cdf7d14490bbdde38b4b38b057 P eb285e93c1b841d799d40d78773c908e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:18.272835 7875 raft_consensus.cc:2454] T aac7e3cdf7d14490bbdde38b4b38b057 P 1f5942cc327540c3810845302e4aa7ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337faaa1672a48b0b2d658f4c5494cab in term 1.
I20260812 03:03:18.273093 7345 leader_election.cc:302] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f5942cc327540c3810845302e4aa7ae, 337faaa1672a48b0b2d658f4c5494cab; no voters:
I20260812 03:03:18.273348 7662 raft_consensus.cc:2454] T aac7e3cdf7d14490bbdde38b4b38b057 P eb285e93c1b841d799d40d78773c908e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337faaa1672a48b0b2d658f4c5494cab in term 1.
I20260812 03:03:18.273530 8002 raft_consensus.cc:2781] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:18.273615 8002 raft_consensus.cc:686] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab [term 1 LEADER]: Becoming Leader. State: Replica: 337faaa1672a48b0b2d658f4c5494cab, State: Running, Role: LEADER
I20260812 03:03:18.273746 8002 consensus_queue.cc:237] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } }
I20260812 03:03:18.275251 7200 catalog_manager.cc:5495] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab reported cstate change: term changed from 0 to 1, leader changed from <none> to 337faaa1672a48b0b2d658f4c5494cab (127.5.96.129). New cstate: current_term: 1 leader_uuid: "337faaa1672a48b0b2d658f4c5494cab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } health_report { overall_health: UNKNOWN } } }
Aug 12 03:03:18 dist-test-slave-6bbx krb5kdc[7133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503796, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:18.309932 8035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:18.301212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55336 (local address 127.5.96.190:45861)
0812 03:03:18.301468 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:18.301476 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:18.301494 (+ 18us) server_negotiation.cc:407] Connection header received
0812 03:03:18.301610 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:18.301614 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:18.301662 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:18.301767 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:18.302376 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.303208 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.303996 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.304354 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.304526 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:18.306777 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:18.306780 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:18.306782 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:18.306803 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:18.308119 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:18.308658 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:18.308660 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:18.308661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:18.308732 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:18.309005 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:18.309007 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:18.309008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:18.309158 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:18.309232 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:18.309558 (+ 326us) server_negotiation.cc:299] Negotiation successful
0812 03:03:18.309745 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":84,"server-negotiator.queue_time_us":191,"thread_start_us":99,"threads_started":1}
I20260812 03:03:18.309935 8034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:18.301132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:45861 (local address 127.0.0.1:55336)
0812 03:03:18.301419 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:18.301433 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:18.301516 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:18.301785 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:18.301787 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:18.302157 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:18.302252 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:18.302258 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.303305 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.303307 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:18.303894 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:18.303903 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.304529 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.304531 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:18.304590 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:18.305198 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:18.305209 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:18.306645 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:18.308233 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:18.308242 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:18.308248 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:18.308570 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:18.308811 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:18.308815 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:18.308817 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:18.308924 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:18.309259 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:18.309266 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:18.309394 (+ 128us) client_negotiation.cc:769] Sending connection context
0812 03:03:18.309520 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 03:03:18.309684 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":105,"threads_started":1}
I20260812 03:03:18.314034 7200 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:55336:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 03:03:18.314808 7200 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:03:18.319708 7421 tablet_service.cc:1467] Processing CreateTablet for tablet f5b8147334fd4ab1be594c023256bc90 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0d1c514f76b84de1a0d08f1423dce58a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:03:18.320122 7421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5b8147334fd4ab1be594c023256bc90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:18.319729 7634 tablet_service.cc:1467] Processing CreateTablet for tablet f5b8147334fd4ab1be594c023256bc90 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0d1c514f76b84de1a0d08f1423dce58a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:03:18.321406 7634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5b8147334fd4ab1be594c023256bc90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:18.323063 7841 tablet_service.cc:1467] Processing CreateTablet for tablet f5b8147334fd4ab1be594c023256bc90 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0d1c514f76b84de1a0d08f1423dce58a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:03:18.323421 7841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5b8147334fd4ab1be594c023256bc90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:18.324438 7980 tablet_bootstrap.cc:492] T f5b8147334fd4ab1be594c023256bc90 P eb285e93c1b841d799d40d78773c908e: Bootstrap starting.
I20260812 03:03:18.325559 7980 tablet_bootstrap.cc:654] T f5b8147334fd4ab1be594c023256bc90 P eb285e93c1b841d799d40d78773c908e: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:18.326094 7976 tablet_bootstrap.cc:492] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae: Bootstrap starting.
I20260812 03:03:18.326493 7977 tablet_bootstrap.cc:492] T f5b8147334fd4ab1be594c023256bc90 P 337faaa1672a48b0b2d658f4c5494cab: Bootstrap starting.
I20260812 03:03:18.327047 7976 tablet_bootstrap.cc:654] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:18.327464 7977 tablet_bootstrap.cc:654] T f5b8147334fd4ab1be594c023256bc90 P 337faaa1672a48b0b2d658f4c5494cab: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:18.328058 7976 tablet_bootstrap.cc:492] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae: No bootstrap required, opened a new log
I20260812 03:03:18.328125 7976 ts_tablet_manager.cc:1397] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:03:18.328476 7976 raft_consensus.cc:348] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } }
I20260812 03:03:18.328572 7976 raft_consensus.cc:374] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:18.328603 7976 raft_consensus.cc:729] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f5942cc327540c3810845302e4aa7ae, State: Initialized, Role: FOLLOWER
I20260812 03:03:18.328722 7976 consensus_queue.cc:260] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } }
I20260812 03:03:18.328961 7976 ts_tablet_manager.cc:1428] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:03:18.329010 7980 tablet_bootstrap.cc:492] T f5b8147334fd4ab1be594c023256bc90 P eb285e93c1b841d799d40d78773c908e: No bootstrap required, opened a new log
I20260812 03:03:18.329103 7980 ts_tablet_manager.cc:1397] T f5b8147334fd4ab1be594c023256bc90 P eb285e93c1b841d799d40d78773c908e: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 03:03:18.329253 7985 tablet_replica.cc:439] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } } }
I20260812 03:03:18.329373 7985 tablet_replica.cc:442] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:03:18.329519 7980 raft_consensus.cc:348] T f5b8147334fd4ab1be594c023256bc90 P eb285e93c1b841d799d40d78773c908e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } }
I20260812 03:03:18.329643 7980 raft_consensus.cc:374] T f5b8147334fd4ab1be594c023256bc90 P eb285e93c1b841d799d40d78773c908e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:18.329685 7980 raft_consensus.cc:729] T f5b8147334fd4ab1be594c023256bc90 P eb285e93c1b841d799d40d78773c908e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb285e93c1b841d799d40d78773c908e, State: Initialized, Role: FOLLOWER
I20260812 03:03:18.329813 7980 consensus_queue.cc:260] T f5b8147334fd4ab1be594c023256bc90 P eb285e93c1b841d799d40d78773c908e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } }
I20260812 03:03:18.330077 7988 tablet_replica.cc:439] T f5b8147334fd4ab1be594c023256bc90 P eb285e93c1b841d799d40d78773c908e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } } }
I20260812 03:03:18.330224 7988 tablet_replica.cc:442] T f5b8147334fd4ab1be594c023256bc90 P eb285e93c1b841d799d40d78773c908e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:03:18.330852 7980 ts_tablet_manager.cc:1428] T f5b8147334fd4ab1be594c023256bc90 P eb285e93c1b841d799d40d78773c908e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:03:18.331650 7977 tablet_bootstrap.cc:492] T f5b8147334fd4ab1be594c023256bc90 P 337faaa1672a48b0b2d658f4c5494cab: No bootstrap required, opened a new log
I20260812 03:03:18.331768 7977 ts_tablet_manager.cc:1397] T f5b8147334fd4ab1be594c023256bc90 P 337faaa1672a48b0b2d658f4c5494cab: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260812 03:03:18.332195 7977 raft_consensus.cc:348] T f5b8147334fd4ab1be594c023256bc90 P 337faaa1672a48b0b2d658f4c5494cab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } }
I20260812 03:03:18.332317 7977 raft_consensus.cc:374] T f5b8147334fd4ab1be594c023256bc90 P 337faaa1672a48b0b2d658f4c5494cab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:18.332351 7977 raft_consensus.cc:729] T f5b8147334fd4ab1be594c023256bc90 P 337faaa1672a48b0b2d658f4c5494cab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337faaa1672a48b0b2d658f4c5494cab, State: Initialized, Role: FOLLOWER
I20260812 03:03:18.332474 7977 consensus_queue.cc:260] T f5b8147334fd4ab1be594c023256bc90 P 337faaa1672a48b0b2d658f4c5494cab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } }
I20260812 03:03:18.332722 7991 tablet_replica.cc:439] T f5b8147334fd4ab1be594c023256bc90 P 337faaa1672a48b0b2d658f4c5494cab: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } } }
I20260812 03:03:18.332791 7977 ts_tablet_manager.cc:1428] T f5b8147334fd4ab1be594c023256bc90 P 337faaa1672a48b0b2d658f4c5494cab: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 03:03:18.332847 7991 tablet_replica.cc:442] T f5b8147334fd4ab1be594c023256bc90 P 337faaa1672a48b0b2d658f4c5494cab: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:03:18.341588 7985 raft_consensus.cc:482] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:03:18.341691 7985 raft_consensus.cc:504] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } }
I20260812 03:03:18.342854 7985 leader_election.cc:288] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337faaa1672a48b0b2d658f4c5494cab (127.5.96.129:33987), eb285e93c1b841d799d40d78773c908e (127.5.96.130:35107)
I20260812 03:03:18.347151 7973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:18.343630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:59033 (local address 127.5.96.130:35107)
0812 03:03:18.343759 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:18.343763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:18.344004 (+ 241us) server_negotiation.cc:407] Connection header received
0812 03:03:18.344095 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:18.344098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:18.344141 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:18.344209 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:18.344462 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.344998 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.346044 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.346619 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.346733 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:18.346786 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:18.346967 (+ 181us) server_negotiation.cc:299] Negotiation successful
0812 03:03:18.347037 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 03:03:18.347172 8043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:18.343376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:35107 (local address 127.5.96.131:59033)
0812 03:03:18.343919 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:18.343931 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:18.344023 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:18.344230 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:18.344233 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:18.344250 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:18.344367 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:18.344372 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.345097 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.345100 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:18.345929 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:18.345939 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.346735 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.346739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:18.346844 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:18.346870 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 03:03:18.346943 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 03:03:18.347017 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"mutex_wait_us":3,"thread_start_us":93,"threads_started":1}
I20260812 03:03:18.347386 7965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:18.343162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:57849 (local address 127.5.96.129:33987)
0812 03:03:18.343273 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:18.343277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:18.343368 (+ 91us) server_negotiation.cc:407] Connection header received
0812 03:03:18.343545 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:18.343548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:18.343595 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:18.343686 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:18.343927 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.344609 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.346195 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.346758 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.346856 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:18.346893 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:18.347182 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 03:03:18.347275 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 03:03:18.347790 8042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:18.343012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:33987 (local address 127.5.96.131:57849)
0812 03:03:18.343285 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:18.343299 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:18.343435 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:18.343719 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:18.343723 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:18.343740 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:18.343832 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:18.343839 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.344717 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.344721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:18.346067 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:18.346078 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.346950 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.346954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:18.347043 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:18.347065 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:03:18.347514 (+ 449us) client_negotiation.cc:240] Negotiation successful
0812 03:03:18.347586 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":41,"thread_start_us":100,"threads_started":1}
I20260812 03:03:18.347923 7662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5b8147334fd4ab1be594c023256bc90" candidate_uuid: "1f5942cc327540c3810845302e4aa7ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb285e93c1b841d799d40d78773c908e" is_pre_election: true
I20260812 03:03:18.348034 7662 raft_consensus.cc:2454] T f5b8147334fd4ab1be594c023256bc90 P eb285e93c1b841d799d40d78773c908e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f5942cc327540c3810845302e4aa7ae in term 0.
I20260812 03:03:18.348218 7452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5b8147334fd4ab1be594c023256bc90" candidate_uuid: "1f5942cc327540c3810845302e4aa7ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337faaa1672a48b0b2d658f4c5494cab" is_pre_election: true
I20260812 03:03:18.348413 7452 raft_consensus.cc:2454] T f5b8147334fd4ab1be594c023256bc90 P 337faaa1672a48b0b2d658f4c5494cab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f5942cc327540c3810845302e4aa7ae in term 0.
I20260812 03:03:18.348538 7767 leader_election.cc:302] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f5942cc327540c3810845302e4aa7ae, eb285e93c1b841d799d40d78773c908e; no voters:
I20260812 03:03:18.348824 7985 raft_consensus.cc:2781] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:03:18.348886 7985 raft_consensus.cc:482] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:03:18.348924 7985 raft_consensus.cc:3037] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:18.349773 7985 raft_consensus.cc:504] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } }
I20260812 03:03:18.350023 7985 leader_election.cc:288] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae [CANDIDATE]: Term 1 election: Requested vote from peers 337faaa1672a48b0b2d658f4c5494cab (127.5.96.129:33987), eb285e93c1b841d799d40d78773c908e (127.5.96.130:35107)
I20260812 03:03:18.350402 7662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5b8147334fd4ab1be594c023256bc90" candidate_uuid: "1f5942cc327540c3810845302e4aa7ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb285e93c1b841d799d40d78773c908e"
I20260812 03:03:18.350394 7452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5b8147334fd4ab1be594c023256bc90" candidate_uuid: "1f5942cc327540c3810845302e4aa7ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337faaa1672a48b0b2d658f4c5494cab"
I20260812 03:03:18.350497 7452 raft_consensus.cc:3037] T f5b8147334fd4ab1be594c023256bc90 P 337faaa1672a48b0b2d658f4c5494cab [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:18.350500 7662 raft_consensus.cc:3037] T f5b8147334fd4ab1be594c023256bc90 P eb285e93c1b841d799d40d78773c908e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:18.351277 7662 raft_consensus.cc:2454] T f5b8147334fd4ab1be594c023256bc90 P eb285e93c1b841d799d40d78773c908e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f5942cc327540c3810845302e4aa7ae in term 1.
I20260812 03:03:18.351512 7767 leader_election.cc:302] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f5942cc327540c3810845302e4aa7ae, eb285e93c1b841d799d40d78773c908e; no voters:
I20260812 03:03:18.351538 7452 raft_consensus.cc:2454] T f5b8147334fd4ab1be594c023256bc90 P 337faaa1672a48b0b2d658f4c5494cab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f5942cc327540c3810845302e4aa7ae in term 1.
I20260812 03:03:18.351701 7985 raft_consensus.cc:2781] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:18.351897 7985 raft_consensus.cc:686] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae [term 1 LEADER]: Becoming Leader. State: Replica: 1f5942cc327540c3810845302e4aa7ae, State: Running, Role: LEADER
I20260812 03:03:18.352119 7985 consensus_queue.cc:237] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } }
I20260812 03:03:18.353498 8045 tablet_replica.cc:439] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae: TxnStatusTablet state changed. Reason: New leader 1f5942cc327540c3810845302e4aa7ae. Latest consensus state: current_term: 1 leader_uuid: "1f5942cc327540c3810845302e4aa7ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } } }
I20260812 03:03:18.353654 8045 tablet_replica.cc:442] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:03:18.354137 8048 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 03:03:18.354316 8048 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 03:03:18.354542 7198 catalog_manager.cc:5495] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f5942cc327540c3810845302e4aa7ae (127.5.96.131). New cstate: current_term: 1 leader_uuid: "1f5942cc327540c3810845302e4aa7ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } health_report { overall_health: HEALTHY } } }
I20260812 03:03:18.372977 7967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:18.369308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46754 (local address 127.5.96.131:38573)
0812 03:03:18.369419 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:18.369423 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:18.369437 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:03:18.369578 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:18.369581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:18.369637 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:18.369737 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:03:18.369941 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.370725 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.371490 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.371856 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.371955 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:18.372187 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:03:18.372644 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:03:18.372747 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:18.372799 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 03:03:18.372850 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:03:18.373008 8034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:18.369225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:38573 (local address 127.0.0.1:46754)
0812 03:03:18.369365 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:18.369380 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:18.369463 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:18.369737 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:18.369739 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:18.369746 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:03:18.369829 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:18.369833 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.370810 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.370812 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:18.371396 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:18.371402 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.371956 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.371959 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:18.372027 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:18.372071 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:03:18.372725 (+ 654us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:03:18.372736 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 03:03:18.372784 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 03:03:18.372894 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 03:03:18.375581 7662 raft_consensus.cc:1261] T f5b8147334fd4ab1be594c023256bc90 P eb285e93c1b841d799d40d78773c908e [term 1 FOLLOWER]: Refusing update from remote peer 1f5942cc327540c3810845302e4aa7ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:18.375736 7452 raft_consensus.cc:1261] T f5b8147334fd4ab1be594c023256bc90 P 337faaa1672a48b0b2d658f4c5494cab [term 1 FOLLOWER]: Refusing update from remote peer 1f5942cc327540c3810845302e4aa7ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:18.376112 8045 consensus_queue.cc:1035] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:18.376281 7985 consensus_queue.cc:1035] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:18.377842 7991 tablet_replica.cc:439] T f5b8147334fd4ab1be594c023256bc90 P 337faaa1672a48b0b2d658f4c5494cab: TxnStatusTablet state changed. Reason: New leader 1f5942cc327540c3810845302e4aa7ae. Latest consensus state: current_term: 1 leader_uuid: "1f5942cc327540c3810845302e4aa7ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } } }
I20260812 03:03:18.377872 7988 tablet_replica.cc:439] T f5b8147334fd4ab1be594c023256bc90 P eb285e93c1b841d799d40d78773c908e: TxnStatusTablet state changed. Reason: New leader 1f5942cc327540c3810845302e4aa7ae. Latest consensus state: current_term: 1 leader_uuid: "1f5942cc327540c3810845302e4aa7ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } } }
I20260812 03:03:18.377947 7991 tablet_replica.cc:442] T f5b8147334fd4ab1be594c023256bc90 P 337faaa1672a48b0b2d658f4c5494cab: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:03:18.377975 7988 tablet_replica.cc:442] T f5b8147334fd4ab1be594c023256bc90 P eb285e93c1b841d799d40d78773c908e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:03:18.379163 8045 tablet_replica.cc:439] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1f5942cc327540c3810845302e4aa7ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } } }
I20260812 03:03:18.379257 8045 tablet_replica.cc:442] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:03:18.380084 7988 tablet_replica.cc:439] T f5b8147334fd4ab1be594c023256bc90 P eb285e93c1b841d799d40d78773c908e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f5942cc327540c3810845302e4aa7ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } } }
I20260812 03:03:18.380183 7988 tablet_replica.cc:442] T f5b8147334fd4ab1be594c023256bc90 P eb285e93c1b841d799d40d78773c908e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:03:18.385249 8045 tablet_replica.cc:439] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1f5942cc327540c3810845302e4aa7ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } } }
I20260812 03:03:18.385350 8045 tablet_replica.cc:442] T f5b8147334fd4ab1be594c023256bc90 P 1f5942cc327540c3810845302e4aa7ae: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:03:18.386376 7991 tablet_replica.cc:439] T f5b8147334fd4ab1be594c023256bc90 P 337faaa1672a48b0b2d658f4c5494cab: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f5942cc327540c3810845302e4aa7ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337faaa1672a48b0b2d658f4c5494cab" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33987 } } peers { permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 } } peers { permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 } } }
I20260812 03:03:18.386478 7991 tablet_replica.cc:442] T f5b8147334fd4ab1be594c023256bc90 P 337faaa1672a48b0b2d658f4c5494cab: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:03:18.394420 8064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:18.390888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:38573 (local address 127.5.96.129:39005)
0812 03:03:18.391250 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:18.391262 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:18.391349 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:18.391558 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:18.391560 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:18.391565 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:03:18.391652 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:18.391656 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.392630 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.392634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:18.393293 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:18.393303 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.393820 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.393823 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:18.393886 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:18.393928 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:03:18.394173 (+ 245us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:03:18.394177 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:03:18.394216 (+ 39us) client_negotiation.cc:240] Negotiation successful
0812 03:03:18.394292 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":96,"threads_started":1}
I20260812 03:03:18.394418 7967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:18.390974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:39005 (local address 127.5.96.131:38573)
0812 03:03:18.391092 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:18.391097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:18.391327 (+ 230us) server_negotiation.cc:407] Connection header received
0812 03:03:18.391424 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:18.391426 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:18.391473 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:18.391554 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:03:18.391758 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.392504 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.393398 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.393721 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.393825 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:18.394034 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:03:18.394105 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:03:18.394170 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:18.394235 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 03:03:18.394280 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:03:18.400460 7662 raft_consensus.cc:1261] T aac7e3cdf7d14490bbdde38b4b38b057 P eb285e93c1b841d799d40d78773c908e [term 1 FOLLOWER]: Refusing update from remote peer 337faaa1672a48b0b2d658f4c5494cab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:18.400594 7875 raft_consensus.cc:1261] T aac7e3cdf7d14490bbdde38b4b38b057 P 1f5942cc327540c3810845302e4aa7ae [term 1 FOLLOWER]: Refusing update from remote peer 337faaa1672a48b0b2d658f4c5494cab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:18.400866 8002 consensus_queue.cc:1035] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb285e93c1b841d799d40d78773c908e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:18.401000 7991 consensus_queue.cc:1035] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f5942cc327540c3810845302e4aa7ae" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:18.403443 7994 mvcc.cc:205] Tried to move back new op lower bound from 7317519558244163584 to 7317519557730435072. Current Snapshot: MvccSnapshot[applied={T|T < 7317519558244163584}]
I20260812 03:03:18.422232 7965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:18.418707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:40123 (local address 127.5.96.129:33987)
0812 03:03:18.418816 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:18.418820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:18.418992 (+ 172us) server_negotiation.cc:407] Connection header received
0812 03:03:18.419079 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:18.419081 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:18.419129 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:18.419208 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:03:18.419382 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.420157 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.421213 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.421472 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.421570 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:18.421831 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:03:18.421900 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:03:18.421955 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:18.422040 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 03:03:18.422106 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:03:18.422261 8069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:18.418624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:33987 (local address 127.5.96.131:40123)
0812 03:03:18.418923 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:18.418935 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:18.419015 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:18.419209 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:18.419211 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:18.419216 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:03:18.419307 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:18.419311 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.420263 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.420266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:18.421106 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:18.421116 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.421569 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.421573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:18.421665 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:18.421707 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:03:18.421970 (+ 263us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:03:18.421976 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:03:18.422032 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 03:03:18.422117 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":94,"threads_started":1}
I20260812 03:03:18.446101 7200 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:18.446239 7200 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:18.448210 7200 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 59e293ad27564221b1ec6b76ebeb7de5: Sending DeleteTablet for 3 replicas of tablet aac7e3cdf7d14490bbdde38b4b38b057
I20260812 03:03:18.448671 7421 tablet_service.cc:1514] Processing DeleteTablet for tablet aac7e3cdf7d14490bbdde38b4b38b057 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:18 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:57754
I20260812 03:03:18.448846 7841 tablet_service.cc:1514] Processing DeleteTablet for tablet aac7e3cdf7d14490bbdde38b4b38b057 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:18 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:46738
I20260812 03:03:18.448876 7634 tablet_service.cc:1514] Processing DeleteTablet for tablet aac7e3cdf7d14490bbdde38b4b38b057 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:18 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:56752
I20260812 03:03:18.449045 8014 tablet_replica.cc:330] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab: stopping tablet replica
I20260812 03:03:18.449047 8015 tablet_replica.cc:330] T aac7e3cdf7d14490bbdde38b4b38b057 P 1f5942cc327540c3810845302e4aa7ae: stopping tablet replica
I20260812 03:03:18.449174 8015 raft_consensus.cc:2229] T aac7e3cdf7d14490bbdde38b4b38b057 P 1f5942cc327540c3810845302e4aa7ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:18.449177 8014 raft_consensus.cc:2229] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:03:18.449280 8015 raft_consensus.cc:2258] T aac7e3cdf7d14490bbdde38b4b38b057 P 1f5942cc327540c3810845302e4aa7ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:18.449337 8014 raft_consensus.cc:2258] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:18.449502 8017 tablet_replica.cc:330] T aac7e3cdf7d14490bbdde38b4b38b057 P eb285e93c1b841d799d40d78773c908e: stopping tablet replica
I20260812 03:03:18.449623 8017 raft_consensus.cc:2229] T aac7e3cdf7d14490bbdde38b4b38b057 P eb285e93c1b841d799d40d78773c908e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:18.449744 8017 raft_consensus.cc:2258] T aac7e3cdf7d14490bbdde38b4b38b057 P eb285e93c1b841d799d40d78773c908e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:18.450066 8015 ts_tablet_manager.cc:1905] T aac7e3cdf7d14490bbdde38b4b38b057 P 1f5942cc327540c3810845302e4aa7ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:18.450565 8014 ts_tablet_manager.cc:1905] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:18.452845 8015 ts_tablet_manager.cc:1918] T aac7e3cdf7d14490bbdde38b4b38b057 P 1f5942cc327540c3810845302e4aa7ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:03:18.452912 8015 log.cc:1196] T aac7e3cdf7d14490bbdde38b4b38b057 P 1f5942cc327540c3810845302e4aa7ae: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-2/wal/wals/aac7e3cdf7d14490bbdde38b4b38b057
I20260812 03:03:18.453169 8015 ts_tablet_manager.cc:1939] T aac7e3cdf7d14490bbdde38b4b38b057 P 1f5942cc327540c3810845302e4aa7ae: Deleting consensus metadata
I20260812 03:03:18.453804 7180 catalog_manager.cc:4841] TS 1f5942cc327540c3810845302e4aa7ae (127.5.96.131:38573): tablet aac7e3cdf7d14490bbdde38b4b38b057 (table test-table [id=5be4e835233f422f8943482c519597d6]) successfully deleted
I20260812 03:03:18.454419 8014 ts_tablet_manager.cc:1918] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:03:18.454495 8014 log.cc:1196] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-0/wal/wals/aac7e3cdf7d14490bbdde38b4b38b057
I20260812 03:03:18.454728 8014 ts_tablet_manager.cc:1939] T aac7e3cdf7d14490bbdde38b4b38b057 P 337faaa1672a48b0b2d658f4c5494cab: Deleting consensus metadata
I20260812 03:03:18.455341 7179 catalog_manager.cc:4841] TS 337faaa1672a48b0b2d658f4c5494cab (127.5.96.129:33987): tablet aac7e3cdf7d14490bbdde38b4b38b057 (table test-table [id=5be4e835233f422f8943482c519597d6]) successfully deleted
I20260812 03:03:18.455658 8017 ts_tablet_manager.cc:1905] T aac7e3cdf7d14490bbdde38b4b38b057 P eb285e93c1b841d799d40d78773c908e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:18.458240 8017 ts_tablet_manager.cc:1918] T aac7e3cdf7d14490bbdde38b4b38b057 P eb285e93c1b841d799d40d78773c908e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:03:18.458326 8017 log.cc:1196] T aac7e3cdf7d14490bbdde38b4b38b057 P eb285e93c1b841d799d40d78773c908e: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786503790637001-5506-0/minicluster-data/ts-1/wal/wals/aac7e3cdf7d14490bbdde38b4b38b057
I20260812 03:03:18.458544 8017 ts_tablet_manager.cc:1939] T aac7e3cdf7d14490bbdde38b4b38b057 P eb285e93c1b841d799d40d78773c908e: Deleting consensus metadata
I20260812 03:03:18.459156 7179 catalog_manager.cc:4841] TS eb285e93c1b841d799d40d78773c908e (127.5.96.130:35107): tablet aac7e3cdf7d14490bbdde38b4b38b057 (table test-table [id=5be4e835233f422f8943482c519597d6]) successfully deleted
Aug 12 03:03:18 dist-test-slave-6bbx krb5kdc[7133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503797, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.96.129@KRBTEST.COM
I20260812 03:03:18.474895 7965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:18.456049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57776 (local address 127.5.96.129:33987)
0812 03:03:18.456157 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:18.456161 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:18.463966 (+ 7805us) server_negotiation.cc:407] Connection header received
0812 03:03:18.464150 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:18.464153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:18.464196 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:18.464241 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:18.465149 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.465931 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.467116 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.467440 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.467550 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:18.471374 (+ 3824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:18.471386 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:18.471400 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:18.471429 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:18.473284 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:18.473687 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:18.473694 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:18.473696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:18.473791 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:18.474023 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:18.474026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:18.474028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:18.474325 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:18.474527 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:18.474617 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 03:03:18.474739 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260812 03:03:18.475418 7505 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57776
I20260812 03:03:18.484500 8035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:18.477056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55344 (local address 127.5.96.190:45861)
0812 03:03:18.477188 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:18.477193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:18.477245 (+ 52us) server_negotiation.cc:407] Connection header received
0812 03:03:18.477337 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:18.477339 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:18.477380 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:18.477463 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:18.477991 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.478743 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.479532 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:18.479916 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:18.480027 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:18.481336 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:18.481341 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:18.481343 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:18.481370 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:18.482882 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:18.483253 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:18.483257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:18.483259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:18.483359 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:18.483583 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:18.483586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:18.483588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:18.483851 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:18.483988 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:18.484188 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 03:03:18.484316 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260812 03:03:18.484867 7200 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55344
I20260812 03:03:18.486130 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 7317
I20260812 03:03:18.501204 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 7526
I20260812 03:03:18.516577 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 7738
I20260812 03:03:18.527619 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 7155
2026-08-12T03:03:18Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2990 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:03:18 dist-test-slave-6bbx krb5kdc[8082](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:03:18 dist-test-slave-6bbx krb5kdc[8082](info): set up 2 sockets
Aug 12 03:03:18 dist-test-slave-6bbx krb5kdc[8082](info): commencing operation
krb5kdc: starting...
W20260812 03:03:19.672044 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:03:19 dist-test-slave-6bbx krb5kdc[8082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:03:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:03:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:03:19.843374 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:42729
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:39391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:42729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:19.952522 8101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:19.952760 8101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:19.952814 8101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:19.956718 8101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:03:19.956768 8101 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 03:03:19.956789 8101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:19.956811 8101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:19.956833 8101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:03:19.956861 8101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:03:19.961874 8101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:39391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:42729
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:42729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:19.962807 8101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:19.963759 8101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:19.971514 8109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:19.971577 8106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:19.971685 8107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:19.971792 8101 server_base.cc:1034] running on GCE node
I20260812 03:03:19.972296 8101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:19.973282 8101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:19.974433 8101 hybrid_clock.cc:648] HybridClock initialized: now 1786503799974421 us; error 28 us; skew 500 ppm
Aug 12 03:03:19 dist-test-slave-6bbx krb5kdc[8082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503799, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:19.976922 8101 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:03:19.977720 8101 webserver.cc:460] Webserver started at http://127.5.96.190:40849/ using document root <none> and password file <none>
I20260812 03:03:19.978201 8101 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:19.978273 8101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:19.978468 8101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:19.980427 8101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "1fdf03fd2d9248a9a9e0c1e02a414010"
format_stamp: "Formatted at 2026-08-12 03:03:19 on dist-test-slave-6bbx"
server_key: "961caf95fc2342c86c6a8657afdb242f"
server_key_iv: "ad700796fff2a6d72ef091d1f1b746b4"
server_key_version: "encryptionkey@0"
I20260812 03:03:19.980952 8101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "1fdf03fd2d9248a9a9e0c1e02a414010"
format_stamp: "Formatted at 2026-08-12 03:03:19 on dist-test-slave-6bbx"
server_key: "961caf95fc2342c86c6a8657afdb242f"
server_key_iv: "ad700796fff2a6d72ef091d1f1b746b4"
server_key_version: "encryptionkey@0"
I20260812 03:03:19.984346 8101 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:03:19.986472 8118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:19.987558 8101 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:19.987694 8101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "1fdf03fd2d9248a9a9e0c1e02a414010"
format_stamp: "Formatted at 2026-08-12 03:03:19 on dist-test-slave-6bbx"
server_key: "961caf95fc2342c86c6a8657afdb242f"
server_key_iv: "ad700796fff2a6d72ef091d1f1b746b4"
server_key_version: "encryptionkey@0"
I20260812 03:03:19.987790 8101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:19.997845 8101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:20.000273 8101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:20.000427 8101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:20.007651 8101 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:42729
I20260812 03:03:20.007658 8207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:42729 every 8 connection(s)
I20260812 03:03:20.008522 8101 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:03:20.009213 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 8101
I20260812 03:03:20.009338 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:03:20.009604 5506 external_mini_cluster.cc:1428] Setting key bc3685bfd60968e24640ac7d85f10e05
I20260812 03:03:20.011511 8209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:20.017678 8209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fdf03fd2d9248a9a9e0c1e02a414010: Bootstrap starting.
Aug 12 03:03:20 dist-test-slave-6bbx krb5kdc[8082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:20.019733 8209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fdf03fd2d9248a9a9e0c1e02a414010: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:20.020854 8209 log.cc:824] T 00000000000000000000000000000000 P 1fdf03fd2d9248a9a9e0c1e02a414010: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:20.022612 8209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fdf03fd2d9248a9a9e0c1e02a414010: No bootstrap required, opened a new log
I20260812 03:03:20.022662 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:20.010823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45978 (local address 127.5.96.190:42729)
0812 03:03:20.011195 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:20.011209 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:20.011244 (+ 35us) server_negotiation.cc:407] Connection header received
0812 03:03:20.011906 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:20.011923 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:20.012228 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:20.012484 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:20.013305 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.014274 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.015358 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.015841 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.015951 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:20.018499 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:20.018504 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:20.018515 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:20.018541 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:20.020118 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:20.020504 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:20.020509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:20.020513 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:20.020636 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:20.020897 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:20.020900 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:20.020901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:20.021282 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:20.021457 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:20.021747 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 03:03:20.022001 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":111,"threads_started":1}
I20260812 03:03:20.025691 8209 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fdf03fd2d9248a9a9e0c1e02a414010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fdf03fd2d9248a9a9e0c1e02a414010" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42729 } }
I20260812 03:03:20.025902 8209 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fdf03fd2d9248a9a9e0c1e02a414010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:20.025959 8209 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fdf03fd2d9248a9a9e0c1e02a414010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf03fd2d9248a9a9e0c1e02a414010, State: Initialized, Role: FOLLOWER
I20260812 03:03:20.026326 8209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fdf03fd2d9248a9a9e0c1e02a414010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fdf03fd2d9248a9a9e0c1e02a414010" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42729 } }
I20260812 03:03:20.026449 8209 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1fdf03fd2d9248a9a9e0c1e02a414010 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:03:20.026511 8209 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fdf03fd2d9248a9a9e0c1e02a414010 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:03:20.026607 8209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fdf03fd2d9248a9a9e0c1e02a414010 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:20.027469 8209 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fdf03fd2d9248a9a9e0c1e02a414010 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fdf03fd2d9248a9a9e0c1e02a414010" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42729 } }
I20260812 03:03:20.027819 8209 leader_election.cc:302] T 00000000000000000000000000000000 P 1fdf03fd2d9248a9a9e0c1e02a414010 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1fdf03fd2d9248a9a9e0c1e02a414010; no voters:
I20260812 03:03:20.028084 8209 leader_election.cc:288] T 00000000000000000000000000000000 P 1fdf03fd2d9248a9a9e0c1e02a414010 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:03:20.028173 8217 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fdf03fd2d9248a9a9e0c1e02a414010 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:20.028353 8217 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fdf03fd2d9248a9a9e0c1e02a414010 [term 1 LEADER]: Becoming Leader. State: Replica: 1fdf03fd2d9248a9a9e0c1e02a414010, State: Running, Role: LEADER
I20260812 03:03:20.028662 8217 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fdf03fd2d9248a9a9e0c1e02a414010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fdf03fd2d9248a9a9e0c1e02a414010" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42729 } }
I20260812 03:03:20.029153 8209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fdf03fd2d9248a9a9e0c1e02a414010 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:03:20.030094 8218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fdf03fd2d9248a9a9e0c1e02a414010 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fdf03fd2d9248a9a9e0c1e02a414010" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fdf03fd2d9248a9a9e0c1e02a414010" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42729 } } }
I20260812 03:03:20.030216 8218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fdf03fd2d9248a9a9e0c1e02a414010 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:20.030503 8220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fdf03fd2d9248a9a9e0c1e02a414010 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fdf03fd2d9248a9a9e0c1e02a414010. Latest consensus state: current_term: 1 leader_uuid: "1fdf03fd2d9248a9a9e0c1e02a414010" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fdf03fd2d9248a9a9e0c1e02a414010" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42729 } } }
I20260812 03:03:20.030598 8220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fdf03fd2d9248a9a9e0c1e02a414010 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:20.030696 8228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:03:20.033620 8228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:03:20.038165 8228 catalog_manager.cc:1347] Generated new cluster ID: 07c9d6bc0277467e98d848bf67ff3adb
I20260812 03:03:20.038226 8228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:03:20.058971 8228 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:03:20.059803 8228 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:03:20.069219 8228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fdf03fd2d9248a9a9e0c1e02a414010: Generated new TSK 0
I20260812 03:03:20.069762 8228 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.96.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
I20260812 03:03:20.118019 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:0
--local_ip_for_outbound_sockets=127.5.96.129
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:42729
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:39391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:20.226816 8247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:20.227037 8247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:20.227080 8247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:20.231151 8247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:20.231201 8247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:20.231244 8247 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 03:03:20.231271 8247 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:03:20.231312 8247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:03:20.236371 8247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:39391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.96.190:42729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:20.237236 8247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:20.238122 8247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:20.246821 8254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:20.246821 8256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:20.246913 8262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:20.247118 8247 server_base.cc:1034] running on GCE node
I20260812 03:03:20.247484 8247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:20.248087 8247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:20.249243 8247 hybrid_clock.cc:648] HybridClock initialized: now 1786503800249223 us; error 30 us; skew 500 ppm
Aug 12 03:03:20 dist-test-slave-6bbx krb5kdc[8082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503800, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:20.251765 8247 init.cc:377] Logged in from keytab as kudu/127.5.96.129@KRBTEST.COM (short username kudu)
I20260812 03:03:20.252522 8247 webserver.cc:460] Webserver started at http://127.5.96.129:38441/ using document root <none> and password file <none>
I20260812 03:03:20.253031 8247 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:20.253098 8247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:20.253276 8247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:20.255100 8247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-0/data/instance:
uuid: "f0d3c2ba5dbf4158879e784e90767ffd"
format_stamp: "Formatted at 2026-08-12 03:03:20 on dist-test-slave-6bbx"
server_key: "89faea19dbd2d7787ad33ff9f7e8c0f3"
server_key_iv: "fd0a4712f3b99194ebf48f7591ba1e44"
server_key_version: "encryptionkey@0"
I20260812 03:03:20.255610 8247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance:
uuid: "f0d3c2ba5dbf4158879e784e90767ffd"
format_stamp: "Formatted at 2026-08-12 03:03:20 on dist-test-slave-6bbx"
server_key: "89faea19dbd2d7787ad33ff9f7e8c0f3"
server_key_iv: "fd0a4712f3b99194ebf48f7591ba1e44"
server_key_version: "encryptionkey@0"
I20260812 03:03:20.258936 8247 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:03:20.261118 8271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:20.262187 8247 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:03:20.262291 8247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "f0d3c2ba5dbf4158879e784e90767ffd"
format_stamp: "Formatted at 2026-08-12 03:03:20 on dist-test-slave-6bbx"
server_key: "89faea19dbd2d7787ad33ff9f7e8c0f3"
server_key_iv: "fd0a4712f3b99194ebf48f7591ba1e44"
server_key_version: "encryptionkey@0"
I20260812 03:03:20.262375 8247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:20.283908 8247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:20.286265 8247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:20.286422 8247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:20.287722 8247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:20.287771 8247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:20.287820 8247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:20.287846 8247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:03:20 dist-test-slave-6bbx krb5kdc[8082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503800, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:20.297832 8247 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:39297
I20260812 03:03:20.297945 8451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:39297 every 8 connection(s)
I20260812 03:03:20.298822 8247 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:03:20.300804 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:20.288436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:45727 (local address 127.5.96.190:42729)
0812 03:03:20.288545 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:20.288549 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:20.289492 (+ 943us) server_negotiation.cc:407] Connection header received
0812 03:03:20.290337 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:20.290340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:20.290398 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:20.290469 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:20.291811 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.292579 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.293765 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.294078 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.294170 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:20.297460 (+ 3290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:20.297464 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:20.297467 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:20.297496 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:20.299003 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:20.299561 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:20.299564 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:20.299566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:20.299645 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:20.299967 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:20.299969 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:20.299970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:20.300129 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:20.300202 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:20.300577 (+ 375us) server_negotiation.cc:299] Negotiation successful
0812 03:03:20.300678 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 03:03:20.301460 8286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:20.288900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:42729 (local address 127.5.96.129:45727)
0812 03:03:20.289360 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:20.289399 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:20.290127 (+ 728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:20.290742 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:20.290763 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:20.291184 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:20.291656 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:20.291692 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.292688 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.292691 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:20.293649 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:20.293659 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.294234 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.294238 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:20.294387 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:20.294878 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:20.294894 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:20.297283 (+ 2389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:20.299113 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:20.299118 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:20.299127 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:20.299441 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:20.299750 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:20.299753 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:20.299756 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:20.299890 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:20.300204 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:20.300208 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:20.300406 (+ 198us) client_negotiation.cc:769] Sending connection context
0812 03:03:20.300546 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 03:03:20.300777 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":142,"threads_started":1}
I20260812 03:03:20.302546 8452 heartbeater.cc:347] Connected to a master server at 127.5.96.190:42729
I20260812 03:03:20.302675 8452 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:20.303097 8452 heartbeater.cc:511] Master 127.5.96.190:42729 requested a full tablet report, sending...
I20260812 03:03:20.304167 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 8247
I20260812 03:03:20.304260 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance
I20260812 03:03:20.304555 5506 external_mini_cluster.cc:1428] Setting key a3d0c033f1f8fd5250f915d3ddc2ead9
I20260812 03:03:20.304642 8139 ts_manager.cc:194] Registered new tserver with Master: f0d3c2ba5dbf4158879e784e90767ffd (127.5.96.129:39297)
I20260812 03:03:20.306192 8139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.129@KRBTEST.COM'} at 127.5.96.129:45727
I20260812 03:03:20.311267 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:20.304767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:43957 (local address 127.5.96.190:42729)
0812 03:03:20.304873 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:20.304877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:20.304891 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:03:20.304927 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:20.304930 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:20.304966 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:20.305026 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:20.305488 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.306227 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.307266 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.307485 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.307569 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:20.308559 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:20.308563 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:20.308565 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:20.308590 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:20.309797 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:20.310173 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:20.310176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:20.310178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:20.310249 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:20.310491 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:20.310494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:20.310496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:20.310698 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:20.310785 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:20.311027 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 03:03:20.311138 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:03:20.311275 8460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:20.304486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:42729 (local address 127.5.96.129:43957)
0812 03:03:20.304773 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:20.304786 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:20.304868 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:20.305056 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:20.305059 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:20.305247 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:20.305355 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:20.305363 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.306388 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.306391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:20.307158 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:20.307167 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.307581 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.307585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:20.307768 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:20.308147 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:20.308157 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:20.308435 (+ 278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:20.309899 (+ 1464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:20.309902 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:20.309904 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:20.310085 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:20.310331 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:20.310334 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:20.310336 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:20.310413 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:20.310788 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:20.310791 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:20.310877 (+ 86us) client_negotiation.cc:769] Sending connection context
0812 03:03:20.310962 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 03:03:20.311088 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":20,"thread_start_us":110,"threads_started":1}
WARNING: no policy specified for kudu/127.5.96.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
I20260812 03:03:20.328789 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:0
--local_ip_for_outbound_sockets=127.5.96.130
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:42729
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:39391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:20.435529 8466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:20.435814 8466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:20.435884 8466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:20.439785 8466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:20.439832 8466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:20.439884 8466 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 03:03:20.439910 8466 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:03:20.439961 8466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:03:20.445011 8466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:39391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.96.190:42729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:20.445941 8466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:20.446880 8466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:20.455785 8478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:20.455790 8476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:20.455788 8475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:20.455981 8466 server_base.cc:1034] running on GCE node
I20260812 03:03:20.456387 8466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:20.456944 8466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:20.458076 8466 hybrid_clock.cc:648] HybridClock initialized: now 1786503800458046 us; error 42 us; skew 500 ppm
Aug 12 03:03:20 dist-test-slave-6bbx krb5kdc[8082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503800, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:20.460812 8466 init.cc:377] Logged in from keytab as kudu/127.5.96.130@KRBTEST.COM (short username kudu)
I20260812 03:03:20.461612 8466 webserver.cc:460] Webserver started at http://127.5.96.130:42363/ using document root <none> and password file <none>
I20260812 03:03:20.462121 8466 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:20.462178 8466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:20.462379 8466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:20.464314 8466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-1/data/instance:
uuid: "263ca1115c074b4b9414fb436a041eb3"
format_stamp: "Formatted at 2026-08-12 03:03:20 on dist-test-slave-6bbx"
server_key: "201de6f17ac8162a11cf9a16ed473a32"
server_key_iv: "3a0a95961b4f84f7964a8f73cacebc05"
server_key_version: "encryptionkey@0"
I20260812 03:03:20.464843 8466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance:
uuid: "263ca1115c074b4b9414fb436a041eb3"
format_stamp: "Formatted at 2026-08-12 03:03:20 on dist-test-slave-6bbx"
server_key: "201de6f17ac8162a11cf9a16ed473a32"
server_key_iv: "3a0a95961b4f84f7964a8f73cacebc05"
server_key_version: "encryptionkey@0"
I20260812 03:03:20.468391 8466 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:03:20.471047 8490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:20.472226 8466 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:03:20.472337 8466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "263ca1115c074b4b9414fb436a041eb3"
format_stamp: "Formatted at 2026-08-12 03:03:20 on dist-test-slave-6bbx"
server_key: "201de6f17ac8162a11cf9a16ed473a32"
server_key_iv: "3a0a95961b4f84f7964a8f73cacebc05"
server_key_version: "encryptionkey@0"
I20260812 03:03:20.472438 8466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:20.486845 8466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:20.489596 8466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:20.489728 8466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:20.491022 8466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:20.491070 8466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:20.491109 8466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:20.491138 8466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:03:20 dist-test-slave-6bbx krb5kdc[8082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503800, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:20.501170 8466 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:38621
I20260812 03:03:20.501314 8666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:38621 every 8 connection(s)
I20260812 03:03:20.502025 8466 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:03:20.503816 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:20.491744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:51847 (local address 127.5.96.190:42729)
0812 03:03:20.491893 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:20.491897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:20.492862 (+ 965us) server_negotiation.cc:407] Connection header received
0812 03:03:20.493919 (+ 1057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:20.493923 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:20.493984 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:20.494043 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:20.495723 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.496368 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.497454 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.497680 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.497769 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:20.500224 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:20.500228 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:20.500231 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:20.500264 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:20.501703 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:20.502290 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:20.502293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:20.502295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:20.502374 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:20.502741 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:20.502745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:20.502747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:20.502925 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:20.503020 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:20.503493 (+ 473us) server_negotiation.cc:299] Negotiation successful
0812 03:03:20.503600 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 03:03:20.503954 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 8466
I20260812 03:03:20.504061 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance
I20260812 03:03:20.504362 5506 external_mini_cluster.cc:1428] Setting key 0a37ccdb50e23c003be5b03cc76d1018
I20260812 03:03:20.504594 8500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:20.492112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:42729 (local address 127.5.96.130:51847)
0812 03:03:20.492572 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:20.492626 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:20.493654 (+ 1028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:20.494454 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:20.494463 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:20.494901 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:20.495537 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:20.495558 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.496487 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.496491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:20.497331 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:20.497341 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.497831 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.497835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:20.497977 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:20.498449 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:20.498465 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:20.500047 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:20.501815 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:20.501821 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:20.501833 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:20.502192 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:20.502468 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:20.502471 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:20.502473 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:20.502634 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:20.503078 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:20.503083 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:20.503289 (+ 206us) client_negotiation.cc:769] Sending connection context
0812 03:03:20.503435 (+ 146us) client_negotiation.cc:240] Negotiation successful
0812 03:03:20.503704 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":104,"threads_started":1}
I20260812 03:03:20.505638 8668 heartbeater.cc:347] Connected to a master server at 127.5.96.190:42729
I20260812 03:03:20.505784 8668 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:20.506217 8668 heartbeater.cc:511] Master 127.5.96.190:42729 requested a full tablet report, sending...
I20260812 03:03:20.507143 8139 ts_manager.cc:194] Registered new tserver with Master: 263ca1115c074b4b9414fb436a041eb3 (127.5.96.130:38621)
I20260812 03:03:20.507712 8139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.130@KRBTEST.COM'} at 127.5.96.130:51847
I20260812 03:03:20.514599 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:20.507849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:42391 (local address 127.5.96.190:42729)
0812 03:03:20.507957 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:20.507961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:20.508337 (+ 376us) server_negotiation.cc:407] Connection header received
0812 03:03:20.508440 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:20.508443 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:20.508486 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:20.508563 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:20.508996 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.509651 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.510620 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.510814 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.510876 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:20.511940 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:20.511943 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:20.511945 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:20.511964 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:20.513131 (+ 1167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:20.513562 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:20.513564 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:20.513565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:20.513630 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:20.513878 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:20.513879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:20.513881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:20.514053 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:20.514123 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:20.514412 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 03:03:20.514500 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:03:20.514664 8675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:20.507999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:42729 (local address 127.5.96.130:42391)
0812 03:03:20.508245 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:20.508258 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:20.508369 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:20.508585 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:20.508587 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:20.508774 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:20.508883 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:20.508889 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.509773 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.509776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:20.510521 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:20.510533 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.510906 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.510910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:20.510999 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:20.511399 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:20.511413 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:20.511822 (+ 409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:20.513244 (+ 1422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:20.513247 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:20.513249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:20.513479 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:20.513703 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:20.513706 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:20.513708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:20.513803 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:20.514143 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:20.514146 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:20.514254 (+ 108us) client_negotiation.cc:769] Sending connection context
0812 03:03:20.514359 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 03:03:20.514485 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":79,"threads_started":1}
WARNING: no policy specified for kudu/127.5.96.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
I20260812 03:03:20.530066 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:0
--local_ip_for_outbound_sockets=127.5.96.131
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:42729
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:39391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:20.637797 8677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:20.638010 8677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:20.638055 8677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:20.641932 8677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:20.641978 8677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:20.642028 8677 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 03:03:20.642055 8677 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:03:20.642107 8677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:03:20.647104 8677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:39391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.96.190:42729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:20.647984 8677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:20.648872 8677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:20.657725 8682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:20.657719 8683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:20.657722 8686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:20.658048 8677 server_base.cc:1034] running on GCE node
I20260812 03:03:20.658383 8677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:20.658911 8677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:20.660056 8677 hybrid_clock.cc:648] HybridClock initialized: now 1786503800660040 us; error 27 us; skew 500 ppm
Aug 12 03:03:20 dist-test-slave-6bbx krb5kdc[8082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503800, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:20.662544 8677 init.cc:377] Logged in from keytab as kudu/127.5.96.131@KRBTEST.COM (short username kudu)
I20260812 03:03:20.663321 8677 webserver.cc:460] Webserver started at http://127.5.96.131:35599/ using document root <none> and password file <none>
I20260812 03:03:20.663873 8677 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:20.663940 8677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:20.664134 8677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:20.666071 8677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-2/data/instance:
uuid: "1638c83050fa44fc8fcb626feb4ac203"
format_stamp: "Formatted at 2026-08-12 03:03:20 on dist-test-slave-6bbx"
server_key: "55d7d9cd5f972fe25774b3dbf769f4c7"
server_key_iv: "69b2f3521aae92529c71e8867d5c4ef9"
server_key_version: "encryptionkey@0"
I20260812 03:03:20.666590 8677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance:
uuid: "1638c83050fa44fc8fcb626feb4ac203"
format_stamp: "Formatted at 2026-08-12 03:03:20 on dist-test-slave-6bbx"
server_key: "55d7d9cd5f972fe25774b3dbf769f4c7"
server_key_iv: "69b2f3521aae92529c71e8867d5c4ef9"
server_key_version: "encryptionkey@0"
I20260812 03:03:20.670017 8677 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 03:03:20.672209 8702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:20.673273 8677 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:03:20.673368 8677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "1638c83050fa44fc8fcb626feb4ac203"
format_stamp: "Formatted at 2026-08-12 03:03:20 on dist-test-slave-6bbx"
server_key: "55d7d9cd5f972fe25774b3dbf769f4c7"
server_key_iv: "69b2f3521aae92529c71e8867d5c4ef9"
server_key_version: "encryptionkey@0"
I20260812 03:03:20.673458 8677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:20.683135 8677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:20.685492 8677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:20.685642 8677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:20.686919 8677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:20.686966 8677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:20.687002 8677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:20.687031 8677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:03:20 dist-test-slave-6bbx krb5kdc[8082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503800, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:20.696924 8677 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:45757
I20260812 03:03:20.696936 8881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:45757 every 8 connection(s)
I20260812 03:03:20.697926 8677 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:03:20.700209 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:20.687617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:51805 (local address 127.5.96.190:42729)
0812 03:03:20.687787 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:20.687792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:20.688635 (+ 843us) server_negotiation.cc:407] Connection header received
0812 03:03:20.689526 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:20.689529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:20.689588 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:20.689668 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:20.691157 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.692015 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.693717 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.694052 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.694157 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:20.696701 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:20.696705 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:20.696708 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:20.696739 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:20.698241 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:20.698784 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:20.698788 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:20.698790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:20.698869 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:20.699177 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:20.699180 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:20.699182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:20.699354 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:20.699441 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:20.699931 (+ 490us) server_negotiation.cc:299] Negotiation successful
0812 03:03:20.700049 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 03:03:20.700870 8715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:20.688050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:42729 (local address 127.5.96.131:51805)
0812 03:03:20.688479 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:20.688526 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:20.689315 (+ 789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:20.689859 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:20.689868 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:20.690334 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:20.691000 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:20.691023 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.692145 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.692149 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:20.693562 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:20.693574 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.694214 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.694218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:20.694413 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:20.694905 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:20.694924 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:20.696499 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:20.698354 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:20.698360 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:20.698372 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:20.698685 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:20.698952 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:20.698955 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:20.698957 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:20.699084 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:20.699454 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:20.699461 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:20.699690 (+ 229us) client_negotiation.cc:769] Sending connection context
0812 03:03:20.699869 (+ 179us) client_negotiation.cc:240] Negotiation successful
0812 03:03:20.700151 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":103,"threads_started":1}
I20260812 03:03:20.701949 8882 heartbeater.cc:347] Connected to a master server at 127.5.96.190:42729
I20260812 03:03:20.702107 8882 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:20.702546 8882 heartbeater.cc:511] Master 127.5.96.190:42729 requested a full tablet report, sending...
I20260812 03:03:20.703555 8140 ts_manager.cc:194] Registered new tserver with Master: 1638c83050fa44fc8fcb626feb4ac203 (127.5.96.131:45757)
I20260812 03:03:20.704260 8140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.131@KRBTEST.COM'} at 127.5.96.131:51805
I20260812 03:03:20.705382 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 8677
I20260812 03:03:20.705490 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance
I20260812 03:03:20.705755 5506 external_mini_cluster.cc:1428] Setting key 7ffdf3e775bd05c87d5e99f1dd43deed
I20260812 03:03:20.707650 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:03:20.710772 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:20.703965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:59927 (local address 127.5.96.190:42729)
0812 03:03:20.704058 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:20.704061 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:20.704674 (+ 613us) server_negotiation.cc:407] Connection header received
0812 03:03:20.704794 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:20.704797 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:20.704841 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:20.704916 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:20.705310 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.706010 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.706963 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.707168 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.707252 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:20.708244 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:20.708248 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:20.708250 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:20.708273 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:20.709369 (+ 1096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:20.709731 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:20.709735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:20.709737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:20.709814 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:20.710052 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:20.710055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:20.710057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:20.710221 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:20.710302 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:20.710543 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 03:03:20.710636 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 03:03:20.710786 8890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:20.703925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:42729 (local address 127.5.96.131:59927)
0812 03:03:20.704594 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:20.704608 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:20.704709 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:20.704923 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:20.704925 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:20.705079 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:20.705184 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:20.705191 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.706141 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.706144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:20.706866 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:20.706878 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.707264 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.707268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:20.707360 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:20.707821 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:20.707833 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:20.708110 (+ 277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:20.709466 (+ 1356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:20.709469 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:20.709471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:20.709645 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:20.709893 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:20.709896 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:20.709898 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:20.709974 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:20.710310 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:20.710313 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:20.710395 (+ 82us) client_negotiation.cc:769] Sending connection context
0812 03:03:20.710479 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 03:03:20.710604 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":96,"threads_started":1}
Aug 12 03:03:20 dist-test-slave-6bbx krb5kdc[8082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:03:20 dist-test-slave-6bbx krb5kdc[8082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:20.732337 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:20.725470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45984 (local address 127.5.96.190:42729)
0812 03:03:20.725567 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:20.725570 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:20.725580 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:03:20.725608 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:20.725610 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:20.725639 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:20.725696 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:20.726316 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.726764 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.727480 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.727689 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.727772 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:20.729774 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:20.729777 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:20.729779 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:20.729800 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:20.731052 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:20.731383 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:20.731385 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:20.731386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:20.731448 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:20.731651 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:20.731654 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:20.731656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:20.731874 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:20.731976 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:20.732136 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 03:03:20.732234 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 03:03:20.734732 8140 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45984:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:03:20.737577 8140 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:03:20.750759 8908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:20.745009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:45757 (local address 127.0.0.1:44642)
0812 03:03:20.745468 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:20.745511 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:20.745640 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:20.746379 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:20.746385 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:20.746408 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:20.746536 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:20.746544 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.748091 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.748094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:20.749435 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:20.749446 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.750240 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.750244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:20.750361 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:20.750398 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 03:03:20.750500 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 03:03:20.750612 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":87,"spinlock_wait_cycles":5888,"thread_start_us":75,"threads_started":1}
I20260812 03:03:20.750768 8909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:20.745108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44642 (local address 127.5.96.131:45757)
0812 03:03:20.745907 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:20.745918 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:20.745939 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:03:20.746007 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:20.746012 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:20.746204 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:20.746691 (+ 487us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:20.746824 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.747943 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.749567 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.750123 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.750220 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:20.750372 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:20.750538 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 03:03:20.750627 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":708,"thread_start_us":80,"threads_started":1}
I20260812 03:03:20.751148 8914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:20.745141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:38621 (local address 127.0.0.1:35356)
0812 03:03:20.745468 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:20.745513 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:20.745640 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:20.746832 (+ 1192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:20.746835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:20.746854 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:20.746957 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:20.746964 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.748004 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.748007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:20.749575 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:20.749583 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.750730 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.750733 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:20.750821 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:20.750841 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 03:03:20.750897 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 03:03:20.750949 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":66,"spinlock_wait_cycles":1664,"thread_start_us":89,"threads_started":1}
I20260812 03:03:20.751299 8918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:20.746084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35356 (local address 127.5.96.130:38621)
0812 03:03:20.746394 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:20.746404 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:20.746422 (+ 18us) server_negotiation.cc:407] Connection header received
0812 03:03:20.746479 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:20.746482 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:20.746655 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:20.746844 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:20.747058 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.747904 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.749682 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.750449 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.750540 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:20.750676 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:20.751063 (+ 387us) server_negotiation.cc:299] Negotiation successful
0812 03:03:20.751155 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":99,"threads_started":1}
I20260812 03:03:20.751299 8916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:20.745853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43504 (local address 127.5.96.129:39297)
0812 03:03:20.746722 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:20.746734 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:20.746755 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:03:20.746827 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:20.746832 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:20.747003 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:20.747145 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:20.747335 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.748325 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.749518 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.750282 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.750384 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:20.750534 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:20.751096 (+ 562us) server_negotiation.cc:299] Negotiation successful
0812 03:03:20.751174 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":771,"thread_start_us":116,"threads_started":1}
I20260812 03:03:20.750991 8915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:20.745651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:39297 (local address 127.0.0.1:43504)
0812 03:03:20.746060 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:20.746072 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:20.746150 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:20.747117 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:20.747119 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:20.747137 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:20.747229 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:20.747234 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.748421 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.748424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:20.749385 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:20.749394 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:20.750597 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:20.750601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:20.750695 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:20.750718 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:03:20.750795 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 03:03:20.750859 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":1,"thread_start_us":93,"threads_started":1}
I20260812 03:03:20.753784 8775 tablet_service.cc:1467] Processing CreateTablet for tablet f311e19fa8c543c88e87f5f3c4835e73 (DEFAULT_TABLE table=test-table [id=85f3401574a24f048b6925594a1d2695]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:20.754015 8574 tablet_service.cc:1467] Processing CreateTablet for tablet f311e19fa8c543c88e87f5f3c4835e73 (DEFAULT_TABLE table=test-table [id=85f3401574a24f048b6925594a1d2695]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:20.754019 8339 tablet_service.cc:1467] Processing CreateTablet for tablet f311e19fa8c543c88e87f5f3c4835e73 (DEFAULT_TABLE table=test-table [id=85f3401574a24f048b6925594a1d2695]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:20.754787 8775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f311e19fa8c543c88e87f5f3c4835e73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:20.754997 8339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f311e19fa8c543c88e87f5f3c4835e73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:20.754997 8574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f311e19fa8c543c88e87f5f3c4835e73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:20.761421 8919 tablet_bootstrap.cc:492] T f311e19fa8c543c88e87f5f3c4835e73 P 1638c83050fa44fc8fcb626feb4ac203: Bootstrap starting.
I20260812 03:03:20.763062 8922 tablet_bootstrap.cc:492] T f311e19fa8c543c88e87f5f3c4835e73 P 263ca1115c074b4b9414fb436a041eb3: Bootstrap starting.
I20260812 03:03:20.763482 8919 tablet_bootstrap.cc:654] T f311e19fa8c543c88e87f5f3c4835e73 P 1638c83050fa44fc8fcb626feb4ac203: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:20.763979 8920 tablet_bootstrap.cc:492] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd: Bootstrap starting.
I20260812 03:03:20.764240 8919 log.cc:824] T f311e19fa8c543c88e87f5f3c4835e73 P 1638c83050fa44fc8fcb626feb4ac203: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:20.764989 8922 tablet_bootstrap.cc:654] T f311e19fa8c543c88e87f5f3c4835e73 P 263ca1115c074b4b9414fb436a041eb3: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:20.765820 8922 log.cc:824] T f311e19fa8c543c88e87f5f3c4835e73 P 263ca1115c074b4b9414fb436a041eb3: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:20.765901 8919 tablet_bootstrap.cc:492] T f311e19fa8c543c88e87f5f3c4835e73 P 1638c83050fa44fc8fcb626feb4ac203: No bootstrap required, opened a new log
I20260812 03:03:20.765998 8920 tablet_bootstrap.cc:654] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:20.766080 8919 ts_tablet_manager.cc:1397] T f311e19fa8c543c88e87f5f3c4835e73 P 1638c83050fa44fc8fcb626feb4ac203: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260812 03:03:20.766827 8920 log.cc:824] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:20.767632 8922 tablet_bootstrap.cc:492] T f311e19fa8c543c88e87f5f3c4835e73 P 263ca1115c074b4b9414fb436a041eb3: No bootstrap required, opened a new log
I20260812 03:03:20.767834 8922 ts_tablet_manager.cc:1397] T f311e19fa8c543c88e87f5f3c4835e73 P 263ca1115c074b4b9414fb436a041eb3: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260812 03:03:20.768709 8920 tablet_bootstrap.cc:492] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd: No bootstrap required, opened a new log
I20260812 03:03:20.768882 8920 ts_tablet_manager.cc:1397] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 03:03:20.769330 8919 raft_consensus.cc:348] T f311e19fa8c543c88e87f5f3c4835e73 P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } }
I20260812 03:03:20.769532 8919 raft_consensus.cc:374] T f311e19fa8c543c88e87f5f3c4835e73 P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:20.769603 8919 raft_consensus.cc:729] T f311e19fa8c543c88e87f5f3c4835e73 P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1638c83050fa44fc8fcb626feb4ac203, State: Initialized, Role: FOLLOWER
I20260812 03:03:20.770004 8919 consensus_queue.cc:260] T f311e19fa8c543c88e87f5f3c4835e73 P 1638c83050fa44fc8fcb626feb4ac203 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } }
I20260812 03:03:20.770593 8882 heartbeater.cc:503] Master 127.5.96.190:42729 was elected leader, sending a full tablet report...
I20260812 03:03:20.771503 8922 raft_consensus.cc:348] T f311e19fa8c543c88e87f5f3c4835e73 P 263ca1115c074b4b9414fb436a041eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } }
I20260812 03:03:20.771819 8922 raft_consensus.cc:374] T f311e19fa8c543c88e87f5f3c4835e73 P 263ca1115c074b4b9414fb436a041eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:20.771832 8919 ts_tablet_manager.cc:1428] T f311e19fa8c543c88e87f5f3c4835e73 P 1638c83050fa44fc8fcb626feb4ac203: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260812 03:03:20.771924 8922 raft_consensus.cc:729] T f311e19fa8c543c88e87f5f3c4835e73 P 263ca1115c074b4b9414fb436a041eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 263ca1115c074b4b9414fb436a041eb3, State: Initialized, Role: FOLLOWER
I20260812 03:03:20.772384 8922 consensus_queue.cc:260] T f311e19fa8c543c88e87f5f3c4835e73 P 263ca1115c074b4b9414fb436a041eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } }
I20260812 03:03:20.772506 8920 raft_consensus.cc:348] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } }
I20260812 03:03:20.772766 8920 raft_consensus.cc:374] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:20.772820 8920 raft_consensus.cc:729] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0d3c2ba5dbf4158879e784e90767ffd, State: Initialized, Role: FOLLOWER
I20260812 03:03:20.773200 8920 consensus_queue.cc:260] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } }
I20260812 03:03:20.773314 8922 ts_tablet_manager.cc:1428] T f311e19fa8c543c88e87f5f3c4835e73 P 263ca1115c074b4b9414fb436a041eb3: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260812 03:03:20.773636 8668 heartbeater.cc:503] Master 127.5.96.190:42729 was elected leader, sending a full tablet report...
I20260812 03:03:20.774186 8920 ts_tablet_manager.cc:1428] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260812 03:03:20.774504 8452 heartbeater.cc:503] Master 127.5.96.190:42729 was elected leader, sending a full tablet report...
W20260812 03:03:20.799865 8453 tablet.cc:2369] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:03:20.949004 8885 tablet.cc:2369] T f311e19fa8c543c88e87f5f3c4835e73 P 1638c83050fa44fc8fcb626feb4ac203: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:03:21.004635 8669 tablet.cc:2369] T f311e19fa8c543c88e87f5f3c4835e73 P 263ca1115c074b4b9414fb436a041eb3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:03:21.133101 8929 raft_consensus.cc:482] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:03:21.133327 8929 raft_consensus.cc:504] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } }
I20260812 03:03:21.134459 8929 leader_election.cc:288] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1638c83050fa44fc8fcb626feb4ac203 (127.5.96.131:45757), 263ca1115c074b4b9414fb436a041eb3 (127.5.96.130:38621)
I20260812 03:03:21.138254 8918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:21.134809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:60059 (local address 127.5.96.130:38621)
0812 03:03:21.134937 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:21.134942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:21.135188 (+ 246us) server_negotiation.cc:407] Connection header received
0812 03:03:21.135294 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:21.135297 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:21.135351 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:21.135425 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:21.135702 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.136193 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.137332 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.137706 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.137819 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:21.137868 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:21.138084 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 03:03:21.138155 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:03:21.138264 8909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:21.134828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:41603 (local address 127.5.96.131:45757)
0812 03:03:21.134956 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:21.134961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:21.135071 (+ 110us) server_negotiation.cc:407] Connection header received
0812 03:03:21.135193 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:21.135197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:21.135259 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:21.135324 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:21.135658 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.136184 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.137313 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.137694 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.137792 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:21.137847 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:21.138059 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 03:03:21.138130 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:03:21.138273 8932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:21.134687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:38621 (local address 127.5.96.129:60059)
0812 03:03:21.135088 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:21.135102 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:21.135206 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:21.135445 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:21.135449 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:21.135463 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:21.135573 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:21.135578 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.136301 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.136305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:21.137206 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:21.137215 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.137822 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.137826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:21.137966 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:21.137984 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 03:03:21.138054 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 03:03:21.138128 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":141,"thread_start_us":73,"threads_started":1}
I20260812 03:03:21.138283 8930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:21.134687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:45757 (local address 127.5.96.129:41603)
0812 03:03:21.134991 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:21.135003 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:21.135101 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:21.135350 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:21.135352 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:21.135369 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:21.135540 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:21.135552 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.136296 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.136300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:21.137186 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:21.137195 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.137806 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.137810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:21.137934 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:21.137968 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 03:03:21.138025 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 03:03:21.138101 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":78,"thread_start_us":74,"threads_started":1}
I20260812 03:03:21.138823 8811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f311e19fa8c543c88e87f5f3c4835e73" candidate_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1638c83050fa44fc8fcb626feb4ac203" is_pre_election: true
I20260812 03:03:21.139091 8811 raft_consensus.cc:2454] T f311e19fa8c543c88e87f5f3c4835e73 P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0d3c2ba5dbf4158879e784e90767ffd in term 0.
I20260812 03:03:21.139523 8273 leader_election.cc:302] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1638c83050fa44fc8fcb626feb4ac203, f0d3c2ba5dbf4158879e784e90767ffd; no voters:
I20260812 03:03:21.139564 8601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f311e19fa8c543c88e87f5f3c4835e73" candidate_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "263ca1115c074b4b9414fb436a041eb3" is_pre_election: true
I20260812 03:03:21.139784 8929 raft_consensus.cc:2781] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:03:21.139878 8929 raft_consensus.cc:482] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:03:21.139876 8601 raft_consensus.cc:2454] T f311e19fa8c543c88e87f5f3c4835e73 P 263ca1115c074b4b9414fb436a041eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0d3c2ba5dbf4158879e784e90767ffd in term 0.
I20260812 03:03:21.139935 8929 raft_consensus.cc:3037] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:21.141019 8929 raft_consensus.cc:504] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } }
I20260812 03:03:21.141309 8929 leader_election.cc:288] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd [CANDIDATE]: Term 1 election: Requested vote from peers 1638c83050fa44fc8fcb626feb4ac203 (127.5.96.131:45757), 263ca1115c074b4b9414fb436a041eb3 (127.5.96.130:38621)
I20260812 03:03:21.141633 8601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f311e19fa8c543c88e87f5f3c4835e73" candidate_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "263ca1115c074b4b9414fb436a041eb3"
I20260812 03:03:21.141633 8811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f311e19fa8c543c88e87f5f3c4835e73" candidate_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1638c83050fa44fc8fcb626feb4ac203"
I20260812 03:03:21.141750 8601 raft_consensus.cc:3037] T f311e19fa8c543c88e87f5f3c4835e73 P 263ca1115c074b4b9414fb436a041eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:21.141763 8811 raft_consensus.cc:3037] T f311e19fa8c543c88e87f5f3c4835e73 P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:21.142970 8601 raft_consensus.cc:2454] T f311e19fa8c543c88e87f5f3c4835e73 P 263ca1115c074b4b9414fb436a041eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0d3c2ba5dbf4158879e784e90767ffd in term 1.
I20260812 03:03:21.143288 8276 leader_election.cc:302] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 263ca1115c074b4b9414fb436a041eb3, f0d3c2ba5dbf4158879e784e90767ffd; no voters:
I20260812 03:03:21.143399 8811 raft_consensus.cc:2454] T f311e19fa8c543c88e87f5f3c4835e73 P 1638c83050fa44fc8fcb626feb4ac203 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0d3c2ba5dbf4158879e784e90767ffd in term 1.
I20260812 03:03:21.143433 8929 raft_consensus.cc:2781] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:21.143589 8929 raft_consensus.cc:686] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd [term 1 LEADER]: Becoming Leader. State: Replica: f0d3c2ba5dbf4158879e784e90767ffd, State: Running, Role: LEADER
I20260812 03:03:21.143987 8929 consensus_queue.cc:237] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } }
I20260812 03:03:21.146921 8140 catalog_manager.cc:5495] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd reported cstate change: term changed from 0 to 1, leader changed from <none> to f0d3c2ba5dbf4158879e784e90767ffd (127.5.96.129). New cstate: current_term: 1 leader_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } health_report { overall_health: HEALTHY } } }
Aug 12 03:03:21 dist-test-slave-6bbx krb5kdc[8082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503799, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:21.228418 8946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:21.221134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:42729 (local address 127.0.0.1:45992)
0812 03:03:21.221404 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:21.221417 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:21.221507 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:21.221699 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:21.221701 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:21.222077 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:21.222187 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:21.222193 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.222852 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.222854 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:21.223459 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:21.223468 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.223878 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.223881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:21.223940 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:21.224425 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:21.224440 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:21.225756 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:21.227104 (+ 1348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:21.227113 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:21.227117 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:21.227348 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:21.227547 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:21.227548 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:21.227550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:21.227622 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:21.227950 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:21.227954 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:21.228052 (+ 98us) client_negotiation.cc:769] Sending connection context
0812 03:03:21.228138 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 03:03:21.228260 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":112,"threads_started":1}
I20260812 03:03:21.228428 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:21.221187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45992 (local address 127.5.96.190:42729)
0812 03:03:21.221306 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:21.221311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:21.221479 (+ 168us) server_negotiation.cc:407] Connection header received
0812 03:03:21.221573 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:21.221575 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:21.221614 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:21.221677 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:21.222273 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.222780 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.223544 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.223790 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.223859 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:21.225870 (+ 2011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:21.225874 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:21.225876 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:21.225897 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:21.227017 (+ 1120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:21.227418 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:21.227421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:21.227422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:21.227488 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:21.227706 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:21.227708 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:21.227709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:21.227873 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:21.227945 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:21.228178 (+ 233us) server_negotiation.cc:299] Negotiation successful
0812 03:03:21.228303 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":42}
I20260812 03:03:21.231957 8140 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:45992:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 03:03:21.232581 8140 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:03:21.236872 8339 tablet_service.cc:1467] Processing CreateTablet for tablet 34e04144e3464d1292070cd02a5d63cd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=60f18e673fc64e8da3b1f0e27b669fd6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:03:21.236933 8775 tablet_service.cc:1467] Processing CreateTablet for tablet 34e04144e3464d1292070cd02a5d63cd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=60f18e673fc64e8da3b1f0e27b669fd6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:03:21.236941 8574 tablet_service.cc:1467] Processing CreateTablet for tablet 34e04144e3464d1292070cd02a5d63cd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=60f18e673fc64e8da3b1f0e27b669fd6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:03:21.237591 8574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34e04144e3464d1292070cd02a5d63cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:21.237592 8775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34e04144e3464d1292070cd02a5d63cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:21.237592 8339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34e04144e3464d1292070cd02a5d63cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:21.239985 8922 tablet_bootstrap.cc:492] T 34e04144e3464d1292070cd02a5d63cd P 263ca1115c074b4b9414fb436a041eb3: Bootstrap starting.
I20260812 03:03:21.240339 8919 tablet_bootstrap.cc:492] T 34e04144e3464d1292070cd02a5d63cd P 1638c83050fa44fc8fcb626feb4ac203: Bootstrap starting.
I20260812 03:03:21.240895 8922 tablet_bootstrap.cc:654] T 34e04144e3464d1292070cd02a5d63cd P 263ca1115c074b4b9414fb436a041eb3: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:21.241215 8919 tablet_bootstrap.cc:654] T 34e04144e3464d1292070cd02a5d63cd P 1638c83050fa44fc8fcb626feb4ac203: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:21.241786 8920 tablet_bootstrap.cc:492] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd: Bootstrap starting.
I20260812 03:03:21.242074 8922 tablet_bootstrap.cc:492] T 34e04144e3464d1292070cd02a5d63cd P 263ca1115c074b4b9414fb436a041eb3: No bootstrap required, opened a new log
I20260812 03:03:21.242148 8922 ts_tablet_manager.cc:1397] T 34e04144e3464d1292070cd02a5d63cd P 263ca1115c074b4b9414fb436a041eb3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:03:21.242534 8922 raft_consensus.cc:348] T 34e04144e3464d1292070cd02a5d63cd P 263ca1115c074b4b9414fb436a041eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:21.242609 8920 tablet_bootstrap.cc:654] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:21.242614 8922 raft_consensus.cc:374] T 34e04144e3464d1292070cd02a5d63cd P 263ca1115c074b4b9414fb436a041eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:21.242635 8922 raft_consensus.cc:729] T 34e04144e3464d1292070cd02a5d63cd P 263ca1115c074b4b9414fb436a041eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 263ca1115c074b4b9414fb436a041eb3, State: Initialized, Role: FOLLOWER
I20260812 03:03:21.242731 8922 consensus_queue.cc:260] T 34e04144e3464d1292070cd02a5d63cd P 263ca1115c074b4b9414fb436a041eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:21.242923 8922 ts_tablet_manager.cc:1428] T 34e04144e3464d1292070cd02a5d63cd P 263ca1115c074b4b9414fb436a041eb3: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 03:03:21.242965 8928 tablet_replica.cc:439] T 34e04144e3464d1292070cd02a5d63cd P 263ca1115c074b4b9414fb436a041eb3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } } }
I20260812 03:03:21.243113 8928 tablet_replica.cc:442] T 34e04144e3464d1292070cd02a5d63cd P 263ca1115c074b4b9414fb436a041eb3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:03:21.243959 8919 tablet_bootstrap.cc:492] T 34e04144e3464d1292070cd02a5d63cd P 1638c83050fa44fc8fcb626feb4ac203: No bootstrap required, opened a new log
I20260812 03:03:21.244036 8919 ts_tablet_manager.cc:1397] T 34e04144e3464d1292070cd02a5d63cd P 1638c83050fa44fc8fcb626feb4ac203: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 03:03:21.244442 8919 raft_consensus.cc:348] T 34e04144e3464d1292070cd02a5d63cd P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:21.244554 8919 raft_consensus.cc:374] T 34e04144e3464d1292070cd02a5d63cd P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:21.244585 8919 raft_consensus.cc:729] T 34e04144e3464d1292070cd02a5d63cd P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1638c83050fa44fc8fcb626feb4ac203, State: Initialized, Role: FOLLOWER
I20260812 03:03:21.244699 8919 consensus_queue.cc:260] T 34e04144e3464d1292070cd02a5d63cd P 1638c83050fa44fc8fcb626feb4ac203 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:21.244911 8927 tablet_replica.cc:439] T 34e04144e3464d1292070cd02a5d63cd P 1638c83050fa44fc8fcb626feb4ac203: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } } }
I20260812 03:03:21.245062 8927 tablet_replica.cc:442] T 34e04144e3464d1292070cd02a5d63cd P 1638c83050fa44fc8fcb626feb4ac203: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:03:21.245415 8920 tablet_bootstrap.cc:492] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd: No bootstrap required, opened a new log
I20260812 03:03:21.245499 8920 ts_tablet_manager.cc:1397] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 03:03:21.245849 8920 raft_consensus.cc:348] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:21.245956 8920 raft_consensus.cc:374] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:21.245990 8920 raft_consensus.cc:729] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0d3c2ba5dbf4158879e784e90767ffd, State: Initialized, Role: FOLLOWER
I20260812 03:03:21.246098 8920 consensus_queue.cc:260] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:21.246335 8920 ts_tablet_manager.cc:1428] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 03:03:21.246348 8929 tablet_replica.cc:439] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } } }
I20260812 03:03:21.244916 8919 ts_tablet_manager.cc:1428] T 34e04144e3464d1292070cd02a5d63cd P 1638c83050fa44fc8fcb626feb4ac203: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:03:21.246483 8929 tablet_replica.cc:442] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:03:21.302491 8929 raft_consensus.cc:482] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:03:21.302613 8929 raft_consensus.cc:504] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:21.302978 8929 leader_election.cc:288] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 263ca1115c074b4b9414fb436a041eb3 (127.5.96.130:38621), 1638c83050fa44fc8fcb626feb4ac203 (127.5.96.131:45757)
I20260812 03:03:21.303334 8601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34e04144e3464d1292070cd02a5d63cd" candidate_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "263ca1115c074b4b9414fb436a041eb3" is_pre_election: true
I20260812 03:03:21.303354 8811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34e04144e3464d1292070cd02a5d63cd" candidate_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1638c83050fa44fc8fcb626feb4ac203" is_pre_election: true
I20260812 03:03:21.303459 8601 raft_consensus.cc:2454] T 34e04144e3464d1292070cd02a5d63cd P 263ca1115c074b4b9414fb436a041eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0d3c2ba5dbf4158879e784e90767ffd in term 0.
I20260812 03:03:21.303479 8811 raft_consensus.cc:2454] T 34e04144e3464d1292070cd02a5d63cd P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0d3c2ba5dbf4158879e784e90767ffd in term 0.
I20260812 03:03:21.303737 8276 leader_election.cc:302] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 263ca1115c074b4b9414fb436a041eb3, f0d3c2ba5dbf4158879e784e90767ffd; no voters:
I20260812 03:03:21.303908 8929 raft_consensus.cc:2781] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:03:21.303966 8929 raft_consensus.cc:482] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:03:21.304013 8929 raft_consensus.cc:3037] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:21.304869 8929 raft_consensus.cc:504] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:21.305154 8929 leader_election.cc:288] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd [CANDIDATE]: Term 1 election: Requested vote from peers 263ca1115c074b4b9414fb436a041eb3 (127.5.96.130:38621), 1638c83050fa44fc8fcb626feb4ac203 (127.5.96.131:45757)
I20260812 03:03:21.305486 8811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34e04144e3464d1292070cd02a5d63cd" candidate_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1638c83050fa44fc8fcb626feb4ac203"
I20260812 03:03:21.305508 8601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34e04144e3464d1292070cd02a5d63cd" candidate_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "263ca1115c074b4b9414fb436a041eb3"
I20260812 03:03:21.305593 8811 raft_consensus.cc:3037] T 34e04144e3464d1292070cd02a5d63cd P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:21.305601 8601 raft_consensus.cc:3037] T 34e04144e3464d1292070cd02a5d63cd P 263ca1115c074b4b9414fb436a041eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:21.306509 8601 raft_consensus.cc:2454] T 34e04144e3464d1292070cd02a5d63cd P 263ca1115c074b4b9414fb436a041eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0d3c2ba5dbf4158879e784e90767ffd in term 1.
I20260812 03:03:21.306631 8811 raft_consensus.cc:2454] T 34e04144e3464d1292070cd02a5d63cd P 1638c83050fa44fc8fcb626feb4ac203 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0d3c2ba5dbf4158879e784e90767ffd in term 1.
I20260812 03:03:21.306756 8276 leader_election.cc:302] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 263ca1115c074b4b9414fb436a041eb3, f0d3c2ba5dbf4158879e784e90767ffd; no voters:
I20260812 03:03:21.306919 8929 raft_consensus.cc:2781] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:21.307005 8929 raft_consensus.cc:686] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd [term 1 LEADER]: Becoming Leader. State: Replica: f0d3c2ba5dbf4158879e784e90767ffd, State: Running, Role: LEADER
I20260812 03:03:21.307120 8929 consensus_queue.cc:237] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:21.307730 8933 tablet_replica.cc:439] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd: TxnStatusTablet state changed. Reason: New leader f0d3c2ba5dbf4158879e784e90767ffd. Latest consensus state: current_term: 1 leader_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } } }
I20260812 03:03:21.307837 8933 tablet_replica.cc:442] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:03:21.308046 8954 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 03:03:21.308257 8954 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 03:03:21.308774 8138 catalog_manager.cc:5495] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd reported cstate change: term changed from 0 to 1, leader changed from <none> to f0d3c2ba5dbf4158879e784e90767ffd (127.5.96.129). New cstate: current_term: 1 leader_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } health_report { overall_health: UNKNOWN } } }
I20260812 03:03:21.325101 8957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:21.321301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43508 (local address 127.5.96.129:39297)
0812 03:03:21.321537 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:21.321543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:21.321556 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:03:21.321610 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:21.321612 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:21.321655 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:21.321743 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:03:21.321975 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.322507 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.323737 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.323981 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.324054 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:21.324352 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:03:21.324777 (+ 425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:03:21.324857 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:21.324948 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 03:03:21.324993 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":92,"threads_started":1}
I20260812 03:03:21.325155 8946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:21.321198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:39297 (local address 127.0.0.1:43508)
0812 03:03:21.321371 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:21.321387 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:21.321487 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:21.321768 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:21.321770 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:21.321780 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:03:21.321887 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:21.321893 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.322589 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.322593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:21.323591 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:21.323602 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.324077 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.324081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:21.324170 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:21.324230 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:03:21.324859 (+ 629us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:03:21.324875 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 03:03:21.324941 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 03:03:21.325023 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260812 03:03:21.329911 8811 raft_consensus.cc:1261] T 34e04144e3464d1292070cd02a5d63cd P 1638c83050fa44fc8fcb626feb4ac203 [term 1 FOLLOWER]: Refusing update from remote peer f0d3c2ba5dbf4158879e784e90767ffd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:21.329911 8601 raft_consensus.cc:1261] T 34e04144e3464d1292070cd02a5d63cd P 263ca1115c074b4b9414fb436a041eb3 [term 1 FOLLOWER]: Refusing update from remote peer f0d3c2ba5dbf4158879e784e90767ffd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:21.330531 8929 consensus_queue.cc:1035] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:21.330689 8933 consensus_queue.cc:1035] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:21.335135 8928 tablet_replica.cc:439] T 34e04144e3464d1292070cd02a5d63cd P 263ca1115c074b4b9414fb436a041eb3: TxnStatusTablet state changed. Reason: New leader f0d3c2ba5dbf4158879e784e90767ffd. Latest consensus state: current_term: 1 leader_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } } }
I20260812 03:03:21.335260 8928 tablet_replica.cc:442] T 34e04144e3464d1292070cd02a5d63cd P 263ca1115c074b4b9414fb436a041eb3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:03:21.335209 8927 tablet_replica.cc:439] T 34e04144e3464d1292070cd02a5d63cd P 1638c83050fa44fc8fcb626feb4ac203: TxnStatusTablet state changed. Reason: New leader f0d3c2ba5dbf4158879e784e90767ffd. Latest consensus state: current_term: 1 leader_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } } }
I20260812 03:03:21.335299 8927 tablet_replica.cc:442] T 34e04144e3464d1292070cd02a5d63cd P 1638c83050fa44fc8fcb626feb4ac203: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:03:21.337764 8929 tablet_replica.cc:439] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } } }
I20260812 03:03:21.337863 8929 tablet_replica.cc:442] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:03:21.337807 8933 tablet_replica.cc:439] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } } }
I20260812 03:03:21.337903 8933 tablet_replica.cc:442] T 34e04144e3464d1292070cd02a5d63cd P f0d3c2ba5dbf4158879e784e90767ffd: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:03:21.338183 8927 tablet_replica.cc:439] T 34e04144e3464d1292070cd02a5d63cd P 1638c83050fa44fc8fcb626feb4ac203: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } } }
I20260812 03:03:21.338280 8927 tablet_replica.cc:442] T 34e04144e3464d1292070cd02a5d63cd P 1638c83050fa44fc8fcb626feb4ac203: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:03:21.338953 8928 tablet_replica.cc:439] T 34e04144e3464d1292070cd02a5d63cd P 263ca1115c074b4b9414fb436a041eb3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } } }
I20260812 03:03:21.339051 8928 tablet_replica.cc:442] T 34e04144e3464d1292070cd02a5d63cd P 263ca1115c074b4b9414fb436a041eb3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:03:21.339107 8959 mvcc.cc:205] Tried to move back new op lower bound from 7317519570241470464 to 7317519570155216896. Current Snapshot: MvccSnapshot[applied={T|T < 7317519570241470464}]
I20260812 03:03:21.339946 8961 mvcc.cc:205] Tried to move back new op lower bound from 7317519570241470464 to 7317519570155216896. Current Snapshot: MvccSnapshot[applied={T|T < 7317519570241470464}]
I20260812 03:03:21.340169 8964 mvcc.cc:205] Tried to move back new op lower bound from 7317519570241470464 to 7317519570155216896. Current Snapshot: MvccSnapshot[applied={T|T < 7317519570241470464}]
I20260812 03:03:21.349597 8957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:21.346558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43524 (local address 127.5.96.129:39297)
0812 03:03:21.346661 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:21.346665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:21.346871 (+ 206us) server_negotiation.cc:407] Connection header received
0812 03:03:21.346987 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:21.346990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:21.347031 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:21.347088 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:03:21.347322 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.347845 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.348673 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.348890 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.348974 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:21.349207 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:03:21.349286 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:03:21.349343 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:21.349436 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 03:03:21.349484 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 03:03:21.359225 8957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:21.356285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:39779 (local address 127.5.96.129:39297)
0812 03:03:21.356378 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:21.356382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:21.356573 (+ 191us) server_negotiation.cc:407] Connection header received
0812 03:03:21.356688 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:21.356691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:21.356728 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:21.356785 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:03:21.357001 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.357457 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.358269 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.358475 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.358554 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:21.358837 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:03:21.358898 (+ 61us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:03:21.358954 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:21.359032 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 03:03:21.359091 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":29}
I20260812 03:03:21.359256 8989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:21.356258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:39297 (local address 127.5.96.129:39779)
0812 03:03:21.356501 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:21.356512 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:21.356602 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:21.356798 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:21.356800 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:21.356819 (+ 19us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:03:21.356912 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:21.356917 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.357546 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.357549 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:21.358163 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:21.358170 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.358579 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.358583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:21.358664 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:21.358701 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:03:21.358965 (+ 264us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:03:21.358970 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:03:21.359025 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 03:03:21.359119 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":15,"thread_start_us":93,"threads_started":1}
I20260812 03:03:21.364959 8601 raft_consensus.cc:1261] T f311e19fa8c543c88e87f5f3c4835e73 P 263ca1115c074b4b9414fb436a041eb3 [term 1 FOLLOWER]: Refusing update from remote peer f0d3c2ba5dbf4158879e784e90767ffd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:21.365063 8811 raft_consensus.cc:1261] T f311e19fa8c543c88e87f5f3c4835e73 P 1638c83050fa44fc8fcb626feb4ac203 [term 1 FOLLOWER]: Refusing update from remote peer f0d3c2ba5dbf4158879e784e90767ffd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:21.365325 8970 consensus_queue.cc:1035] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:21.365450 8933 consensus_queue.cc:1035] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:21.402503 8138 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:21.402655 8138 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:21.404778 8138 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1fdf03fd2d9248a9a9e0c1e02a414010: Sending DeleteTablet for 3 replicas of tablet f311e19fa8c543c88e87f5f3c4835e73
I20260812 03:03:21.405454 8345 tablet_service.cc:1514] Processing DeleteTablet for tablet f311e19fa8c543c88e87f5f3c4835e73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:21 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:43504
I20260812 03:03:21.405498 8775 tablet_service.cc:1514] Processing DeleteTablet for tablet f311e19fa8c543c88e87f5f3c4835e73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:21 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:44642
I20260812 03:03:21.405925 8994 tablet_replica.cc:330] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd: stopping tablet replica
I20260812 03:03:21.405956 8995 tablet_replica.cc:330] T f311e19fa8c543c88e87f5f3c4835e73 P 1638c83050fa44fc8fcb626feb4ac203: stopping tablet replica
I20260812 03:03:21.406224 8994 raft_consensus.cc:2229] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:03:21.406383 8995 raft_consensus.cc:2229] T f311e19fa8c543c88e87f5f3c4835e73 P 1638c83050fa44fc8fcb626feb4ac203 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:21.406554 8994 raft_consensus.cc:2258] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:21.406594 8995 raft_consensus.cc:2258] T f311e19fa8c543c88e87f5f3c4835e73 P 1638c83050fa44fc8fcb626feb4ac203 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:21.405963 8574 tablet_service.cc:1514] Processing DeleteTablet for tablet f311e19fa8c543c88e87f5f3c4835e73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:21 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:35356
I20260812 03:03:21.407718 8994 ts_tablet_manager.cc:1905] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:21.407847 8138 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45984:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:03:21.408236 8138 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:03:21.408401 8995 ts_tablet_manager.cc:1905] T f311e19fa8c543c88e87f5f3c4835e73 P 1638c83050fa44fc8fcb626feb4ac203: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:21.409749 8996 tablet_replica.cc:330] T f311e19fa8c543c88e87f5f3c4835e73 P 263ca1115c074b4b9414fb436a041eb3: stopping tablet replica
I20260812 03:03:21.410017 8996 raft_consensus.cc:2229] T f311e19fa8c543c88e87f5f3c4835e73 P 263ca1115c074b4b9414fb436a041eb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:21.410202 8996 raft_consensus.cc:2258] T f311e19fa8c543c88e87f5f3c4835e73 P 263ca1115c074b4b9414fb436a041eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:21.410717 8994 ts_tablet_manager.cc:1918] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:03:21.410812 8994 log.cc:1196] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-0/wal/wals/f311e19fa8c543c88e87f5f3c4835e73
I20260812 03:03:21.411123 8994 ts_tablet_manager.cc:1939] T f311e19fa8c543c88e87f5f3c4835e73 P f0d3c2ba5dbf4158879e784e90767ffd: Deleting consensus metadata
I20260812 03:03:21.411316 8996 ts_tablet_manager.cc:1905] T f311e19fa8c543c88e87f5f3c4835e73 P 263ca1115c074b4b9414fb436a041eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:21.412180 8122 catalog_manager.cc:4841] TS f0d3c2ba5dbf4158879e784e90767ffd (127.5.96.129:39297): tablet f311e19fa8c543c88e87f5f3c4835e73 (table test-table [id=85f3401574a24f048b6925594a1d2695]) successfully deleted
I20260812 03:03:21.413865 8775 tablet_service.cc:1467] Processing CreateTablet for tablet 9a46eea114b64e8094d34276fa3ed326 (DEFAULT_TABLE table=test-table [id=f6c9466623bd493f9991ec20e427dd85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:21.414045 8996 ts_tablet_manager.cc:1918] T f311e19fa8c543c88e87f5f3c4835e73 P 263ca1115c074b4b9414fb436a041eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:03:21.414101 8775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a46eea114b64e8094d34276fa3ed326. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:21.414145 8996 log.cc:1196] T f311e19fa8c543c88e87f5f3c4835e73 P 263ca1115c074b4b9414fb436a041eb3: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-1/wal/wals/f311e19fa8c543c88e87f5f3c4835e73
I20260812 03:03:21.414451 8996 ts_tablet_manager.cc:1939] T f311e19fa8c543c88e87f5f3c4835e73 P 263ca1115c074b4b9414fb436a041eb3: Deleting consensus metadata
I20260812 03:03:21.415541 8995 ts_tablet_manager.cc:1918] T f311e19fa8c543c88e87f5f3c4835e73 P 1638c83050fa44fc8fcb626feb4ac203: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:03:21.415629 8995 log.cc:1196] T f311e19fa8c543c88e87f5f3c4835e73 P 1638c83050fa44fc8fcb626feb4ac203: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-2/wal/wals/f311e19fa8c543c88e87f5f3c4835e73
I20260812 03:03:21.415944 8995 ts_tablet_manager.cc:1939] T f311e19fa8c543c88e87f5f3c4835e73 P 1638c83050fa44fc8fcb626feb4ac203: Deleting consensus metadata
I20260812 03:03:21.415956 8122 catalog_manager.cc:4841] TS 263ca1115c074b4b9414fb436a041eb3 (127.5.96.130:38621): tablet f311e19fa8c543c88e87f5f3c4835e73 (table test-table [id=85f3401574a24f048b6925594a1d2695]) successfully deleted
I20260812 03:03:21.416239 8574 tablet_service.cc:1467] Processing CreateTablet for tablet 9a46eea114b64e8094d34276fa3ed326 (DEFAULT_TABLE table=test-table [id=f6c9466623bd493f9991ec20e427dd85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:21.416344 8345 tablet_service.cc:1467] Processing CreateTablet for tablet 9a46eea114b64e8094d34276fa3ed326 (DEFAULT_TABLE table=test-table [id=f6c9466623bd493f9991ec20e427dd85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:21.416464 8574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a46eea114b64e8094d34276fa3ed326. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:21.416566 8345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a46eea114b64e8094d34276fa3ed326. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:21.417711 8119 catalog_manager.cc:4841] TS 1638c83050fa44fc8fcb626feb4ac203 (127.5.96.131:45757): tablet f311e19fa8c543c88e87f5f3c4835e73 (table test-table [id=85f3401574a24f048b6925594a1d2695]) successfully deleted
I20260812 03:03:21.418496 8920 tablet_bootstrap.cc:492] T 9a46eea114b64e8094d34276fa3ed326 P f0d3c2ba5dbf4158879e784e90767ffd: Bootstrap starting.
I20260812 03:03:21.418506 8922 tablet_bootstrap.cc:492] T 9a46eea114b64e8094d34276fa3ed326 P 263ca1115c074b4b9414fb436a041eb3: Bootstrap starting.
I20260812 03:03:21.419351 8919 tablet_bootstrap.cc:492] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203: Bootstrap starting.
I20260812 03:03:21.419387 8922 tablet_bootstrap.cc:654] T 9a46eea114b64e8094d34276fa3ed326 P 263ca1115c074b4b9414fb436a041eb3: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:21.419394 8920 tablet_bootstrap.cc:654] T 9a46eea114b64e8094d34276fa3ed326 P f0d3c2ba5dbf4158879e784e90767ffd: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:21.420321 8919 tablet_bootstrap.cc:654] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:21.421159 8920 tablet_bootstrap.cc:492] T 9a46eea114b64e8094d34276fa3ed326 P f0d3c2ba5dbf4158879e784e90767ffd: No bootstrap required, opened a new log
I20260812 03:03:21.421252 8920 ts_tablet_manager.cc:1397] T 9a46eea114b64e8094d34276fa3ed326 P f0d3c2ba5dbf4158879e784e90767ffd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:03:21.421463 8919 tablet_bootstrap.cc:492] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203: No bootstrap required, opened a new log
I20260812 03:03:21.421535 8919 ts_tablet_manager.cc:1397] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 03:03:21.421624 8922 tablet_bootstrap.cc:492] T 9a46eea114b64e8094d34276fa3ed326 P 263ca1115c074b4b9414fb436a041eb3: No bootstrap required, opened a new log
I20260812 03:03:21.421692 8922 ts_tablet_manager.cc:1397] T 9a46eea114b64e8094d34276fa3ed326 P 263ca1115c074b4b9414fb436a041eb3: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260812 03:03:21.421641 8920 raft_consensus.cc:348] T 9a46eea114b64e8094d34276fa3ed326 P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:21.421741 8920 raft_consensus.cc:374] T 9a46eea114b64e8094d34276fa3ed326 P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:21.421772 8920 raft_consensus.cc:729] T 9a46eea114b64e8094d34276fa3ed326 P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0d3c2ba5dbf4158879e784e90767ffd, State: Initialized, Role: FOLLOWER
I20260812 03:03:21.421883 8920 consensus_queue.cc:260] T 9a46eea114b64e8094d34276fa3ed326 P f0d3c2ba5dbf4158879e784e90767ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:21.421944 8919 raft_consensus.cc:348] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:21.422043 8919 raft_consensus.cc:374] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:21.422075 8919 raft_consensus.cc:729] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1638c83050fa44fc8fcb626feb4ac203, State: Initialized, Role: FOLLOWER
I20260812 03:03:21.422063 8922 raft_consensus.cc:348] T 9a46eea114b64e8094d34276fa3ed326 P 263ca1115c074b4b9414fb436a041eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:21.422140 8920 ts_tablet_manager.cc:1428] T 9a46eea114b64e8094d34276fa3ed326 P f0d3c2ba5dbf4158879e784e90767ffd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:03:21.422170 8922 raft_consensus.cc:374] T 9a46eea114b64e8094d34276fa3ed326 P 263ca1115c074b4b9414fb436a041eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:21.422201 8922 raft_consensus.cc:729] T 9a46eea114b64e8094d34276fa3ed326 P 263ca1115c074b4b9414fb436a041eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 263ca1115c074b4b9414fb436a041eb3, State: Initialized, Role: FOLLOWER
I20260812 03:03:21.422210 8919 consensus_queue.cc:260] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:21.422304 8922 consensus_queue.cc:260] T 9a46eea114b64e8094d34276fa3ed326 P 263ca1115c074b4b9414fb436a041eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:21.422463 8919 ts_tablet_manager.cc:1428] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:03:21.422525 8922 ts_tablet_manager.cc:1428] T 9a46eea114b64e8094d34276fa3ed326 P 263ca1115c074b4b9414fb436a041eb3: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:03:21.665262 8927 raft_consensus.cc:482] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:03:21.665390 8927 raft_consensus.cc:504] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:21.666528 8927 leader_election.cc:288] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0d3c2ba5dbf4158879e784e90767ffd (127.5.96.129:39297), 263ca1115c074b4b9414fb436a041eb3 (127.5.96.130:38621)
I20260812 03:03:21.670601 9005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:21.667285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:56533 (local address 127.5.96.130:38621)
0812 03:03:21.667496 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:21.667499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:21.667561 (+ 62us) server_negotiation.cc:407] Connection header received
0812 03:03:21.667654 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:21.667657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:21.667739 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:21.667820 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:21.668059 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.668537 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.669654 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.670052 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.670141 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:21.670182 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:21.670386 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 03:03:21.670462 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":69,"threads_started":1}
I20260812 03:03:21.670591 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:21.667146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:38621 (local address 127.5.96.131:56533)
0812 03:03:21.667477 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:21.667488 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:21.667581 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:21.667832 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:21.667835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:21.667852 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:21.667961 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:21.667967 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.668646 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.668649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:21.669527 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:21.669539 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.670167 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.670171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:21.670264 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:21.670290 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 03:03:21.670358 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 03:03:21.670428 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":58,"thread_start_us":121,"threads_started":1}
I20260812 03:03:21.670706 8957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:21.666843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:56229 (local address 127.5.96.129:39297)
0812 03:03:21.666979 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:21.666983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:21.667120 (+ 137us) server_negotiation.cc:407] Connection header received
0812 03:03:21.667312 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:21.667316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:21.667370 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:21.667441 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:21.667831 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.668511 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.669641 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.670172 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.670296 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:21.670345 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:21.670524 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 03:03:21.670601 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:03:21.670764 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:21.666774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:39297 (local address 127.5.96.131:56229)
0812 03:03:21.667033 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:21.667046 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:21.667163 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:21.667566 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:21.667569 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:21.667592 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:21.667721 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:21.667728 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.668629 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.668633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:21.669526 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:21.669536 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.670307 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.670310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:21.670402 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:21.670425 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:03:21.670498 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 03:03:21.670564 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":63,"thread_start_us":83,"threads_started":1}
I20260812 03:03:21.671195 8601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a46eea114b64e8094d34276fa3ed326" candidate_uuid: "1638c83050fa44fc8fcb626feb4ac203" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "263ca1115c074b4b9414fb436a041eb3" is_pre_election: true
I20260812 03:03:21.671348 8601 raft_consensus.cc:2454] T 9a46eea114b64e8094d34276fa3ed326 P 263ca1115c074b4b9414fb436a041eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1638c83050fa44fc8fcb626feb4ac203 in term 0.
I20260812 03:03:21.671458 8349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a46eea114b64e8094d34276fa3ed326" candidate_uuid: "1638c83050fa44fc8fcb626feb4ac203" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" is_pre_election: true
I20260812 03:03:21.671728 8349 raft_consensus.cc:2454] T 9a46eea114b64e8094d34276fa3ed326 P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1638c83050fa44fc8fcb626feb4ac203 in term 0.
I20260812 03:03:21.671730 8708 leader_election.cc:302] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1638c83050fa44fc8fcb626feb4ac203, 263ca1115c074b4b9414fb436a041eb3; no voters:
I20260812 03:03:21.671972 8927 raft_consensus.cc:2781] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:03:21.672039 8927 raft_consensus.cc:482] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:03:21.672067 8927 raft_consensus.cc:3037] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:21.705675 8927 raft_consensus.cc:504] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:21.706127 8927 leader_election.cc:288] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203 [CANDIDATE]: Term 1 election: Requested vote from peers f0d3c2ba5dbf4158879e784e90767ffd (127.5.96.129:39297), 263ca1115c074b4b9414fb436a041eb3 (127.5.96.130:38621)
I20260812 03:03:21.706642 8601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a46eea114b64e8094d34276fa3ed326" candidate_uuid: "1638c83050fa44fc8fcb626feb4ac203" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "263ca1115c074b4b9414fb436a041eb3"
I20260812 03:03:21.706787 8601 raft_consensus.cc:3037] T 9a46eea114b64e8094d34276fa3ed326 P 263ca1115c074b4b9414fb436a041eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:21.707788 8601 raft_consensus.cc:2454] T 9a46eea114b64e8094d34276fa3ed326 P 263ca1115c074b4b9414fb436a041eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1638c83050fa44fc8fcb626feb4ac203 in term 1.
I20260812 03:03:21.707854 8349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a46eea114b64e8094d34276fa3ed326" candidate_uuid: "1638c83050fa44fc8fcb626feb4ac203" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0d3c2ba5dbf4158879e784e90767ffd"
I20260812 03:03:21.707981 8349 raft_consensus.cc:3037] T 9a46eea114b64e8094d34276fa3ed326 P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:21.708173 8708 leader_election.cc:302] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1638c83050fa44fc8fcb626feb4ac203, 263ca1115c074b4b9414fb436a041eb3; no voters:
I20260812 03:03:21.708354 8927 raft_consensus.cc:2781] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:21.708547 8927 raft_consensus.cc:686] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203 [term 1 LEADER]: Becoming Leader. State: Replica: 1638c83050fa44fc8fcb626feb4ac203, State: Running, Role: LEADER
I20260812 03:03:21.708806 8927 consensus_queue.cc:237] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:21.709205 8349 raft_consensus.cc:2454] T 9a46eea114b64e8094d34276fa3ed326 P f0d3c2ba5dbf4158879e784e90767ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1638c83050fa44fc8fcb626feb4ac203 in term 1.
I20260812 03:03:21.711143 8139 catalog_manager.cc:5495] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1638c83050fa44fc8fcb626feb4ac203 (127.5.96.131). New cstate: current_term: 1 leader_uuid: "1638c83050fa44fc8fcb626feb4ac203" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } health_report { overall_health: HEALTHY } } }
I20260812 03:03:21.785025 9009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:21.781899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44650 (local address 127.5.96.131:45757)
0812 03:03:21.782111 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:21.782114 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:21.782127 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:03:21.782195 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:21.782197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:21.782248 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:21.782348 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:03:21.782521 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.783021 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.783810 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.784017 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.784084 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:21.784289 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:03:21.784737 (+ 448us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:03:21.784833 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:21.784861 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 03:03:21.784913 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":75,"threads_started":1}
I20260812 03:03:21.794471 8957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:21.791657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:42161 (local address 127.5.96.129:39297)
0812 03:03:21.791794 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:21.791799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:21.792016 (+ 217us) server_negotiation.cc:407] Connection header received
0812 03:03:21.792121 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:21.792124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:21.792169 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:21.792228 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:03:21.792466 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.792931 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.793702 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.793900 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.793960 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:21.794168 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:03:21.794226 (+ 58us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:03:21.794279 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:21.794336 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 03:03:21.794373 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:03:21.794509 9016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:21.791561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:39297 (local address 127.5.96.131:42161)
0812 03:03:21.791938 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:21.791951 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:21.792045 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:21.792256 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:21.792259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:21.792266 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:03:21.792377 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:21.792382 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.793015 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.793017 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:21.793615 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:21.793623 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.793972 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.793974 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:21.794041 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:21.794079 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:03:21.794285 (+ 206us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:03:21.794288 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:03:21.794328 (+ 40us) client_negotiation.cc:240] Negotiation successful
0812 03:03:21.794383 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":128,"threads_started":1}
I20260812 03:03:21.799628 8601 raft_consensus.cc:1261] T 9a46eea114b64e8094d34276fa3ed326 P 263ca1115c074b4b9414fb436a041eb3 [term 1 FOLLOWER]: Refusing update from remote peer 1638c83050fa44fc8fcb626feb4ac203: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:21.799784 8349 raft_consensus.cc:1261] T 9a46eea114b64e8094d34276fa3ed326 P f0d3c2ba5dbf4158879e784e90767ffd [term 1 FOLLOWER]: Refusing update from remote peer 1638c83050fa44fc8fcb626feb4ac203: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:21.800091 9007 consensus_queue.cc:1035] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203 [LEADER]: Connected to new peer: Peer: permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:21.800315 8927 consensus_queue.cc:1035] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:21.816051 9009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:21.813329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:58615 (local address 127.5.96.131:45757)
0812 03:03:21.813435 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:21.813440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:21.813452 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:03:21.813488 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:21.813491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:21.813531 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:21.813589 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:03:21.813805 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.814337 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.815071 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.815270 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.815356 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:21.815632 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:03:21.815732 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:03:21.815803 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:21.815879 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 03:03:21.815932 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 03:03:21.816061 8989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:21.813175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:45757 (local address 127.5.96.129:58615)
0812 03:03:21.813313 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:21.813329 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:21.813401 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:21.813606 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:21.813609 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:21.813616 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:03:21.813707 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:21.813713 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.814423 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.814426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:21.814982 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:21.814990 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:21.815395 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:21.815399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:21.815481 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:21.815514 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:03:21.815806 (+ 292us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:03:21.815812 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:03:21.815863 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 03:03:21.815947 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 03:03:21.816993 8961 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 03:03:23.373256 8139 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:23.373399 8139 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:23.375561 8139 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1fdf03fd2d9248a9a9e0c1e02a414010: Sending DeleteTablet for 3 replicas of tablet 9a46eea114b64e8094d34276fa3ed326
I20260812 03:03:23.376111 8345 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a46eea114b64e8094d34276fa3ed326 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:23 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:43504
I20260812 03:03:23.376128 8775 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a46eea114b64e8094d34276fa3ed326 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:23 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:44642
I20260812 03:03:23.376219 8574 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a46eea114b64e8094d34276fa3ed326 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:23 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:35356
I20260812 03:03:23.376528 9063 tablet_replica.cc:330] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203: stopping tablet replica
I20260812 03:03:23.376564 9067 tablet_replica.cc:330] T 9a46eea114b64e8094d34276fa3ed326 P 263ca1115c074b4b9414fb436a041eb3: stopping tablet replica
I20260812 03:03:23.376585 9064 tablet_replica.cc:330] T 9a46eea114b64e8094d34276fa3ed326 P f0d3c2ba5dbf4158879e784e90767ffd: stopping tablet replica
I20260812 03:03:23.376708 9064 raft_consensus.cc:2229] T 9a46eea114b64e8094d34276fa3ed326 P f0d3c2ba5dbf4158879e784e90767ffd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:23.376725 9063 raft_consensus.cc:2229] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:03:23.376776 9067 raft_consensus.cc:2229] T 9a46eea114b64e8094d34276fa3ed326 P 263ca1115c074b4b9414fb436a041eb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:23.376896 9064 raft_consensus.cc:2258] T 9a46eea114b64e8094d34276fa3ed326 P f0d3c2ba5dbf4158879e784e90767ffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:23.376935 9067 raft_consensus.cc:2258] T 9a46eea114b64e8094d34276fa3ed326 P 263ca1115c074b4b9414fb436a041eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:23.377036 9063 raft_consensus.cc:2258] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:23.377621 9064 ts_tablet_manager.cc:1905] T 9a46eea114b64e8094d34276fa3ed326 P f0d3c2ba5dbf4158879e784e90767ffd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:23.377620 9063 ts_tablet_manager.cc:1905] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:23.379433 9067 ts_tablet_manager.cc:1905] T 9a46eea114b64e8094d34276fa3ed326 P 263ca1115c074b4b9414fb436a041eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 12 03:03:23 dist-test-slave-6bbx krb5kdc[8082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503803, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
I20260812 03:03:23.401618 9064 ts_tablet_manager.cc:1918] T 9a46eea114b64e8094d34276fa3ed326 P f0d3c2ba5dbf4158879e784e90767ffd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:03:23.401618 9063 ts_tablet_manager.cc:1918] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:03:23.401714 9063 log.cc:1196] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-2/wal/wals/9a46eea114b64e8094d34276fa3ed326
I20260812 03:03:23.401876 9064 log.cc:1196] T 9a46eea114b64e8094d34276fa3ed326 P f0d3c2ba5dbf4158879e784e90767ffd: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-0/wal/wals/9a46eea114b64e8094d34276fa3ed326
I20260812 03:03:23.402017 9063 ts_tablet_manager.cc:1939] T 9a46eea114b64e8094d34276fa3ed326 P 1638c83050fa44fc8fcb626feb4ac203: Deleting consensus metadata
I20260812 03:03:23.402343 9064 ts_tablet_manager.cc:1939] T 9a46eea114b64e8094d34276fa3ed326 P f0d3c2ba5dbf4158879e784e90767ffd: Deleting consensus metadata
Aug 12 03:03:23 dist-test-slave-6bbx krb5kdc[8082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503803, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:23.413744 9080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:23.402496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45998 (local address 127.5.96.190:42729)
0812 03:03:23.402848 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:23.402856 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:23.402875 (+ 19us) server_negotiation.cc:407] Connection header received
0812 03:03:23.402986 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:23.402989 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:23.403044 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:23.403144 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:23.403847 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:23.404404 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:23.405196 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:23.405431 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:23.405507 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:23.411135 (+ 5628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:23.411138 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:23.411140 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:23.411165 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:23.412408 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:23.412781 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:23.412783 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:23.412784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:23.412855 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:23.413033 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:23.413035 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:23.413036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:23.413195 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:23.413304 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:23.413472 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 03:03:23.413570 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":82,"threads_started":1}
I20260812 03:03:23.416011 8139 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:03:23.416406 8139 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:03:23.420656 8345 tablet_service.cc:1467] Processing CreateTablet for tablet e7e3f480bf184b0d9d6a3731524a31e9 (DEFAULT_TABLE table=test-table [id=28e1b633fbea4d19ad79e39bc0a6dea8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:23.420749 8574 tablet_service.cc:1467] Processing CreateTablet for tablet e7e3f480bf184b0d9d6a3731524a31e9 (DEFAULT_TABLE table=test-table [id=28e1b633fbea4d19ad79e39bc0a6dea8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:23.420842 8775 tablet_service.cc:1467] Processing CreateTablet for tablet e7e3f480bf184b0d9d6a3731524a31e9 (DEFAULT_TABLE table=test-table [id=28e1b633fbea4d19ad79e39bc0a6dea8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:23.420979 8345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7e3f480bf184b0d9d6a3731524a31e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:23.421079 8775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7e3f480bf184b0d9d6a3731524a31e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:23.423470 9081 tablet_bootstrap.cc:492] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203: Bootstrap starting.
I20260812 03:03:23.424357 9082 tablet_bootstrap.cc:492] T e7e3f480bf184b0d9d6a3731524a31e9 P f0d3c2ba5dbf4158879e784e90767ffd: Bootstrap starting.
I20260812 03:03:23.424567 9081 tablet_bootstrap.cc:654] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:23.425153 9082 tablet_bootstrap.cc:654] T e7e3f480bf184b0d9d6a3731524a31e9 P f0d3c2ba5dbf4158879e784e90767ffd: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:23.425627 9081 tablet_bootstrap.cc:492] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203: No bootstrap required, opened a new log
I20260812 03:03:23.425741 9081 ts_tablet_manager.cc:1397] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:03:23.426184 9081 raft_consensus.cc:348] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:23.426321 9082 tablet_bootstrap.cc:492] T e7e3f480bf184b0d9d6a3731524a31e9 P f0d3c2ba5dbf4158879e784e90767ffd: No bootstrap required, opened a new log
I20260812 03:03:23.426334 9081 raft_consensus.cc:374] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:23.426376 9081 raft_consensus.cc:729] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1638c83050fa44fc8fcb626feb4ac203, State: Initialized, Role: FOLLOWER
I20260812 03:03:23.426448 9082 ts_tablet_manager.cc:1397] T e7e3f480bf184b0d9d6a3731524a31e9 P f0d3c2ba5dbf4158879e784e90767ffd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:23.426510 9081 consensus_queue.cc:260] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:23.426836 9081 ts_tablet_manager.cc:1428] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:03:23.426885 9082 raft_consensus.cc:348] T e7e3f480bf184b0d9d6a3731524a31e9 P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:23.426998 9082 raft_consensus.cc:374] T e7e3f480bf184b0d9d6a3731524a31e9 P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:23.427035 9082 raft_consensus.cc:729] T e7e3f480bf184b0d9d6a3731524a31e9 P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0d3c2ba5dbf4158879e784e90767ffd, State: Initialized, Role: FOLLOWER
I20260812 03:03:23.427153 9082 consensus_queue.cc:260] T e7e3f480bf184b0d9d6a3731524a31e9 P f0d3c2ba5dbf4158879e784e90767ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:23.427408 9082 ts_tablet_manager.cc:1428] T e7e3f480bf184b0d9d6a3731524a31e9 P f0d3c2ba5dbf4158879e784e90767ffd: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 03:03:23.430774 8119 catalog_manager.cc:4841] TS 1638c83050fa44fc8fcb626feb4ac203 (127.5.96.131:45757): tablet 9a46eea114b64e8094d34276fa3ed326 (table test-table [id=f6c9466623bd493f9991ec20e427dd85]) successfully deleted
I20260812 03:03:23.433619 8122 catalog_manager.cc:4841] TS f0d3c2ba5dbf4158879e784e90767ffd (127.5.96.129:39297): tablet 9a46eea114b64e8094d34276fa3ed326 (table test-table [id=f6c9466623bd493f9991ec20e427dd85]) successfully deleted
I20260812 03:03:23.438865 8574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7e3f480bf184b0d9d6a3731524a31e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:23.438889 9067 ts_tablet_manager.cc:1918] T 9a46eea114b64e8094d34276fa3ed326 P 263ca1115c074b4b9414fb436a041eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:03:23.438964 9067 log.cc:1196] T 9a46eea114b64e8094d34276fa3ed326 P 263ca1115c074b4b9414fb436a041eb3: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-1/wal/wals/9a46eea114b64e8094d34276fa3ed326
I20260812 03:03:23.439185 9067 ts_tablet_manager.cc:1939] T 9a46eea114b64e8094d34276fa3ed326 P 263ca1115c074b4b9414fb436a041eb3: Deleting consensus metadata
I20260812 03:03:23.440912 9090 tablet_bootstrap.cc:492] T e7e3f480bf184b0d9d6a3731524a31e9 P 263ca1115c074b4b9414fb436a041eb3: Bootstrap starting.
I20260812 03:03:23.441967 9090 tablet_bootstrap.cc:654] T e7e3f480bf184b0d9d6a3731524a31e9 P 263ca1115c074b4b9414fb436a041eb3: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:23.443040 9090 tablet_bootstrap.cc:492] T e7e3f480bf184b0d9d6a3731524a31e9 P 263ca1115c074b4b9414fb436a041eb3: No bootstrap required, opened a new log
I20260812 03:03:23.443112 9090 ts_tablet_manager.cc:1397] T e7e3f480bf184b0d9d6a3731524a31e9 P 263ca1115c074b4b9414fb436a041eb3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:23.443485 9090 raft_consensus.cc:348] T e7e3f480bf184b0d9d6a3731524a31e9 P 263ca1115c074b4b9414fb436a041eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:23.443594 9090 raft_consensus.cc:374] T e7e3f480bf184b0d9d6a3731524a31e9 P 263ca1115c074b4b9414fb436a041eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:23.443619 9090 raft_consensus.cc:729] T e7e3f480bf184b0d9d6a3731524a31e9 P 263ca1115c074b4b9414fb436a041eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 263ca1115c074b4b9414fb436a041eb3, State: Initialized, Role: FOLLOWER
I20260812 03:03:23.443771 9090 consensus_queue.cc:260] T e7e3f480bf184b0d9d6a3731524a31e9 P 263ca1115c074b4b9414fb436a041eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:23.444131 9090 ts_tablet_manager.cc:1428] T e7e3f480bf184b0d9d6a3731524a31e9 P 263ca1115c074b4b9414fb436a041eb3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:03:23.468537 8122 catalog_manager.cc:4841] TS 263ca1115c074b4b9414fb436a041eb3 (127.5.96.130:38621): tablet 9a46eea114b64e8094d34276fa3ed326 (table test-table [id=f6c9466623bd493f9991ec20e427dd85]) successfully deleted
I20260812 03:03:23.627240 9046 raft_consensus.cc:482] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:03:23.627390 9046 raft_consensus.cc:504] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:23.627799 9046 leader_election.cc:288] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0d3c2ba5dbf4158879e784e90767ffd (127.5.96.129:39297), 263ca1115c074b4b9414fb436a041eb3 (127.5.96.130:38621)
I20260812 03:03:23.628197 8349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7e3f480bf184b0d9d6a3731524a31e9" candidate_uuid: "1638c83050fa44fc8fcb626feb4ac203" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" is_pre_election: true
I20260812 03:03:23.628260 8601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7e3f480bf184b0d9d6a3731524a31e9" candidate_uuid: "1638c83050fa44fc8fcb626feb4ac203" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "263ca1115c074b4b9414fb436a041eb3" is_pre_election: true
I20260812 03:03:23.628351 8349 raft_consensus.cc:2454] T e7e3f480bf184b0d9d6a3731524a31e9 P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1638c83050fa44fc8fcb626feb4ac203 in term 0.
I20260812 03:03:23.628414 8601 raft_consensus.cc:2454] T e7e3f480bf184b0d9d6a3731524a31e9 P 263ca1115c074b4b9414fb436a041eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1638c83050fa44fc8fcb626feb4ac203 in term 0.
I20260812 03:03:23.628691 8708 leader_election.cc:302] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1638c83050fa44fc8fcb626feb4ac203, f0d3c2ba5dbf4158879e784e90767ffd; no voters:
I20260812 03:03:23.628862 9046 raft_consensus.cc:2781] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:03:23.628918 9046 raft_consensus.cc:482] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:03:23.628952 9046 raft_consensus.cc:3037] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:23.630035 9046 raft_consensus.cc:504] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:23.630338 9046 leader_election.cc:288] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203 [CANDIDATE]: Term 1 election: Requested vote from peers f0d3c2ba5dbf4158879e784e90767ffd (127.5.96.129:39297), 263ca1115c074b4b9414fb436a041eb3 (127.5.96.130:38621)
I20260812 03:03:23.630633 8349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7e3f480bf184b0d9d6a3731524a31e9" candidate_uuid: "1638c83050fa44fc8fcb626feb4ac203" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0d3c2ba5dbf4158879e784e90767ffd"
I20260812 03:03:23.630709 8601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7e3f480bf184b0d9d6a3731524a31e9" candidate_uuid: "1638c83050fa44fc8fcb626feb4ac203" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "263ca1115c074b4b9414fb436a041eb3"
I20260812 03:03:23.630756 8349 raft_consensus.cc:3037] T e7e3f480bf184b0d9d6a3731524a31e9 P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:23.630823 8601 raft_consensus.cc:3037] T e7e3f480bf184b0d9d6a3731524a31e9 P 263ca1115c074b4b9414fb436a041eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:23.631697 8349 raft_consensus.cc:2454] T e7e3f480bf184b0d9d6a3731524a31e9 P f0d3c2ba5dbf4158879e784e90767ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1638c83050fa44fc8fcb626feb4ac203 in term 1.
I20260812 03:03:23.631815 8601 raft_consensus.cc:2454] T e7e3f480bf184b0d9d6a3731524a31e9 P 263ca1115c074b4b9414fb436a041eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1638c83050fa44fc8fcb626feb4ac203 in term 1.
I20260812 03:03:23.631985 8708 leader_election.cc:302] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1638c83050fa44fc8fcb626feb4ac203, f0d3c2ba5dbf4158879e784e90767ffd; no voters:
I20260812 03:03:23.632141 9046 raft_consensus.cc:2781] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:23.632222 9046 raft_consensus.cc:686] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203 [term 1 LEADER]: Becoming Leader. State: Replica: 1638c83050fa44fc8fcb626feb4ac203, State: Running, Role: LEADER
I20260812 03:03:23.632365 9046 consensus_queue.cc:237] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:23.633872 8138 catalog_manager.cc:5495] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1638c83050fa44fc8fcb626feb4ac203 (127.5.96.131). New cstate: current_term: 1 leader_uuid: "1638c83050fa44fc8fcb626feb4ac203" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } health_report { overall_health: HEALTHY } } }
I20260812 03:03:23.662835 9098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:23.660195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44652 (local address 127.5.96.131:45757)
0812 03:03:23.660439 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:23.660446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:23.660458 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:03:23.660493 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:23.660495 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:23.660532 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:23.660601 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:03:23.660785 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:23.661283 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:23.662041 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:23.662258 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:23.662322 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:23.662501 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:03:23.662562 (+ 61us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:03:23.662606 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:23.662672 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 03:03:23.662715 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":102,"threads_started":1}
I20260812 03:03:23.668099 8349 raft_consensus.cc:1261] T e7e3f480bf184b0d9d6a3731524a31e9 P f0d3c2ba5dbf4158879e784e90767ffd [term 1 FOLLOWER]: Refusing update from remote peer 1638c83050fa44fc8fcb626feb4ac203: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:23.668195 8601 raft_consensus.cc:1261] T e7e3f480bf184b0d9d6a3731524a31e9 P 263ca1115c074b4b9414fb436a041eb3 [term 1 FOLLOWER]: Refusing update from remote peer 1638c83050fa44fc8fcb626feb4ac203: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:23.668445 9051 consensus_queue.cc:1035] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:23.668589 9046 consensus_queue.cc:1035] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203 [LEADER]: Connected to new peer: Peer: permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:23.697014 8138 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:23.697154 8138 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:23.699083 8138 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1fdf03fd2d9248a9a9e0c1e02a414010: Sending DeleteTablet for 3 replicas of tablet e7e3f480bf184b0d9d6a3731524a31e9
I20260812 03:03:23.699654 8574 tablet_service.cc:1514] Processing DeleteTablet for tablet e7e3f480bf184b0d9d6a3731524a31e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:23 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:35356
I20260812 03:03:23.699853 9067 tablet_replica.cc:330] T e7e3f480bf184b0d9d6a3731524a31e9 P 263ca1115c074b4b9414fb436a041eb3: stopping tablet replica
I20260812 03:03:23.699954 9067 raft_consensus.cc:2229] T e7e3f480bf184b0d9d6a3731524a31e9 P 263ca1115c074b4b9414fb436a041eb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:23.700045 9067 raft_consensus.cc:2258] T e7e3f480bf184b0d9d6a3731524a31e9 P 263ca1115c074b4b9414fb436a041eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:23.700213 8775 tablet_service.cc:1514] Processing DeleteTablet for tablet e7e3f480bf184b0d9d6a3731524a31e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:23 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:44642
I20260812 03:03:23.700367 9063 tablet_replica.cc:330] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203: stopping tablet replica
I20260812 03:03:23.700320 8138 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 03:03:23.700476 9063 raft_consensus.cc:2229] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:03:23.700567 8345 tablet_service.cc:1514] Processing DeleteTablet for tablet e7e3f480bf184b0d9d6a3731524a31e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:23 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:43504
I20260812 03:03:23.700640 9063 raft_consensus.cc:2258] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:23.700709 9064 tablet_replica.cc:330] T e7e3f480bf184b0d9d6a3731524a31e9 P f0d3c2ba5dbf4158879e784e90767ffd: stopping tablet replica
W20260812 03:03:23.700747 8138 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:03:23.700801 9064 raft_consensus.cc:2229] T e7e3f480bf184b0d9d6a3731524a31e9 P f0d3c2ba5dbf4158879e784e90767ffd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:23.700878 9064 raft_consensus.cc:2258] T e7e3f480bf184b0d9d6a3731524a31e9 P f0d3c2ba5dbf4158879e784e90767ffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:23.701532 9063 ts_tablet_manager.cc:1905] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:23.701653 9064 ts_tablet_manager.cc:1905] T e7e3f480bf184b0d9d6a3731524a31e9 P f0d3c2ba5dbf4158879e784e90767ffd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:23.704007 9063 ts_tablet_manager.cc:1918] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:03:23.704062 9063 log.cc:1196] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-2/wal/wals/e7e3f480bf184b0d9d6a3731524a31e9
I20260812 03:03:23.704157 9064 ts_tablet_manager.cc:1918] T e7e3f480bf184b0d9d6a3731524a31e9 P f0d3c2ba5dbf4158879e784e90767ffd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:03:23.704226 9064 log.cc:1196] T e7e3f480bf184b0d9d6a3731524a31e9 P f0d3c2ba5dbf4158879e784e90767ffd: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-0/wal/wals/e7e3f480bf184b0d9d6a3731524a31e9
I20260812 03:03:23.704241 9063 ts_tablet_manager.cc:1939] T e7e3f480bf184b0d9d6a3731524a31e9 P 1638c83050fa44fc8fcb626feb4ac203: Deleting consensus metadata
I20260812 03:03:23.704481 9064 ts_tablet_manager.cc:1939] T e7e3f480bf184b0d9d6a3731524a31e9 P f0d3c2ba5dbf4158879e784e90767ffd: Deleting consensus metadata
I20260812 03:03:23.704774 8119 catalog_manager.cc:4841] TS 1638c83050fa44fc8fcb626feb4ac203 (127.5.96.131:45757): tablet e7e3f480bf184b0d9d6a3731524a31e9 (table test-table [id=28e1b633fbea4d19ad79e39bc0a6dea8]) successfully deleted
I20260812 03:03:23.705361 8122 catalog_manager.cc:4841] TS f0d3c2ba5dbf4158879e784e90767ffd (127.5.96.129:39297): tablet e7e3f480bf184b0d9d6a3731524a31e9 (table test-table [id=28e1b633fbea4d19ad79e39bc0a6dea8]) successfully deleted
I20260812 03:03:23.708369 8345 tablet_service.cc:1467] Processing CreateTablet for tablet e3335a53796c445db8285ea8c7e3b76f (DEFAULT_TABLE table=test-table [id=f6c45cb8be004387902972d2eeb600fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:23.708374 8574 tablet_service.cc:1467] Processing CreateTablet for tablet e3335a53796c445db8285ea8c7e3b76f (DEFAULT_TABLE table=test-table [id=f6c45cb8be004387902972d2eeb600fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:23.708586 8345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3335a53796c445db8285ea8c7e3b76f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:23.708626 8574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3335a53796c445db8285ea8c7e3b76f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:23.710318 9090 tablet_bootstrap.cc:492] T e3335a53796c445db8285ea8c7e3b76f P 263ca1115c074b4b9414fb436a041eb3: Bootstrap starting.
I20260812 03:03:23.710691 8775 tablet_service.cc:1467] Processing CreateTablet for tablet e3335a53796c445db8285ea8c7e3b76f (DEFAULT_TABLE table=test-table [id=f6c45cb8be004387902972d2eeb600fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:23.710902 8775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3335a53796c445db8285ea8c7e3b76f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:23.711196 9090 tablet_bootstrap.cc:654] T e3335a53796c445db8285ea8c7e3b76f P 263ca1115c074b4b9414fb436a041eb3: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:23.712255 9090 tablet_bootstrap.cc:492] T e3335a53796c445db8285ea8c7e3b76f P 263ca1115c074b4b9414fb436a041eb3: No bootstrap required, opened a new log
I20260812 03:03:23.712335 9090 ts_tablet_manager.cc:1397] T e3335a53796c445db8285ea8c7e3b76f P 263ca1115c074b4b9414fb436a041eb3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:03:23.712693 9090 raft_consensus.cc:348] T e3335a53796c445db8285ea8c7e3b76f P 263ca1115c074b4b9414fb436a041eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:23.712782 9090 raft_consensus.cc:374] T e3335a53796c445db8285ea8c7e3b76f P 263ca1115c074b4b9414fb436a041eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:23.712812 9090 raft_consensus.cc:729] T e3335a53796c445db8285ea8c7e3b76f P 263ca1115c074b4b9414fb436a041eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 263ca1115c074b4b9414fb436a041eb3, State: Initialized, Role: FOLLOWER
I20260812 03:03:23.712903 9081 tablet_bootstrap.cc:492] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203: Bootstrap starting.
I20260812 03:03:23.712921 9090 consensus_queue.cc:260] T e3335a53796c445db8285ea8c7e3b76f P 263ca1115c074b4b9414fb436a041eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:23.713181 9090 ts_tablet_manager.cc:1428] T e3335a53796c445db8285ea8c7e3b76f P 263ca1115c074b4b9414fb436a041eb3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:03:23.713272 9067 ts_tablet_manager.cc:1905] T e7e3f480bf184b0d9d6a3731524a31e9 P 263ca1115c074b4b9414fb436a041eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:23.713716 9081 tablet_bootstrap.cc:654] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:23.715337 9067 ts_tablet_manager.cc:1918] T e7e3f480bf184b0d9d6a3731524a31e9 P 263ca1115c074b4b9414fb436a041eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:03:23.715409 9067 log.cc:1196] T e7e3f480bf184b0d9d6a3731524a31e9 P 263ca1115c074b4b9414fb436a041eb3: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-1/wal/wals/e7e3f480bf184b0d9d6a3731524a31e9
I20260812 03:03:23.715631 9067 ts_tablet_manager.cc:1939] T e7e3f480bf184b0d9d6a3731524a31e9 P 263ca1115c074b4b9414fb436a041eb3: Deleting consensus metadata
I20260812 03:03:23.716174 8122 catalog_manager.cc:4841] TS 263ca1115c074b4b9414fb436a041eb3 (127.5.96.130:38621): tablet e7e3f480bf184b0d9d6a3731524a31e9 (table test-table [id=28e1b633fbea4d19ad79e39bc0a6dea8]) successfully deleted
I20260812 03:03:23.716382 9082 tablet_bootstrap.cc:492] T e3335a53796c445db8285ea8c7e3b76f P f0d3c2ba5dbf4158879e784e90767ffd: Bootstrap starting.
I20260812 03:03:23.716956 9081 tablet_bootstrap.cc:492] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203: No bootstrap required, opened a new log
I20260812 03:03:23.717036 9081 ts_tablet_manager.cc:1397] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 03:03:23.717056 9082 tablet_bootstrap.cc:654] T e3335a53796c445db8285ea8c7e3b76f P f0d3c2ba5dbf4158879e784e90767ffd: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:23.717392 9081 raft_consensus.cc:348] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:23.717501 9081 raft_consensus.cc:374] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:23.717540 9081 raft_consensus.cc:729] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1638c83050fa44fc8fcb626feb4ac203, State: Initialized, Role: FOLLOWER
I20260812 03:03:23.717657 9081 consensus_queue.cc:260] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:23.717927 9081 ts_tablet_manager.cc:1428] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 03:03:23.718514 9082 tablet_bootstrap.cc:492] T e3335a53796c445db8285ea8c7e3b76f P f0d3c2ba5dbf4158879e784e90767ffd: No bootstrap required, opened a new log
I20260812 03:03:23.718613 9082 ts_tablet_manager.cc:1397] T e3335a53796c445db8285ea8c7e3b76f P f0d3c2ba5dbf4158879e784e90767ffd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:03:23.719002 9082 raft_consensus.cc:348] T e3335a53796c445db8285ea8c7e3b76f P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:23.719113 9082 raft_consensus.cc:374] T e3335a53796c445db8285ea8c7e3b76f P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:23.719153 9082 raft_consensus.cc:729] T e3335a53796c445db8285ea8c7e3b76f P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0d3c2ba5dbf4158879e784e90767ffd, State: Initialized, Role: FOLLOWER
I20260812 03:03:23.719266 9082 consensus_queue.cc:260] T e3335a53796c445db8285ea8c7e3b76f P f0d3c2ba5dbf4158879e784e90767ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:23.719497 9082 ts_tablet_manager.cc:1428] T e3335a53796c445db8285ea8c7e3b76f P f0d3c2ba5dbf4158879e784e90767ffd: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 03:03:23.768899 9023 raft_consensus.cc:482] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:03:23.769006 9023 raft_consensus.cc:504] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:23.769372 9023 leader_election.cc:288] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0d3c2ba5dbf4158879e784e90767ffd (127.5.96.129:39297), 263ca1115c074b4b9414fb436a041eb3 (127.5.96.130:38621)
I20260812 03:03:23.769699 8349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3335a53796c445db8285ea8c7e3b76f" candidate_uuid: "1638c83050fa44fc8fcb626feb4ac203" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" is_pre_election: true
I20260812 03:03:23.769765 8601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3335a53796c445db8285ea8c7e3b76f" candidate_uuid: "1638c83050fa44fc8fcb626feb4ac203" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "263ca1115c074b4b9414fb436a041eb3" is_pre_election: true
I20260812 03:03:23.769827 8349 raft_consensus.cc:2454] T e3335a53796c445db8285ea8c7e3b76f P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1638c83050fa44fc8fcb626feb4ac203 in term 0.
I20260812 03:03:23.769898 8601 raft_consensus.cc:2454] T e3335a53796c445db8285ea8c7e3b76f P 263ca1115c074b4b9414fb436a041eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1638c83050fa44fc8fcb626feb4ac203 in term 0.
I20260812 03:03:23.770092 8708 leader_election.cc:302] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1638c83050fa44fc8fcb626feb4ac203, f0d3c2ba5dbf4158879e784e90767ffd; no voters:
I20260812 03:03:23.770264 9023 raft_consensus.cc:2781] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:03:23.770335 9023 raft_consensus.cc:482] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:03:23.770383 9023 raft_consensus.cc:3037] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:23.771270 9023 raft_consensus.cc:504] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:23.771559 9023 leader_election.cc:288] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203 [CANDIDATE]: Term 1 election: Requested vote from peers f0d3c2ba5dbf4158879e784e90767ffd (127.5.96.129:39297), 263ca1115c074b4b9414fb436a041eb3 (127.5.96.130:38621)
I20260812 03:03:23.771880 8349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3335a53796c445db8285ea8c7e3b76f" candidate_uuid: "1638c83050fa44fc8fcb626feb4ac203" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0d3c2ba5dbf4158879e784e90767ffd"
I20260812 03:03:23.771905 8601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3335a53796c445db8285ea8c7e3b76f" candidate_uuid: "1638c83050fa44fc8fcb626feb4ac203" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "263ca1115c074b4b9414fb436a041eb3"
I20260812 03:03:23.771996 8349 raft_consensus.cc:3037] T e3335a53796c445db8285ea8c7e3b76f P f0d3c2ba5dbf4158879e784e90767ffd [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:23.772001 8601 raft_consensus.cc:3037] T e3335a53796c445db8285ea8c7e3b76f P 263ca1115c074b4b9414fb436a041eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:23.772825 8349 raft_consensus.cc:2454] T e3335a53796c445db8285ea8c7e3b76f P f0d3c2ba5dbf4158879e784e90767ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1638c83050fa44fc8fcb626feb4ac203 in term 1.
I20260812 03:03:23.772836 8601 raft_consensus.cc:2454] T e3335a53796c445db8285ea8c7e3b76f P 263ca1115c074b4b9414fb436a041eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1638c83050fa44fc8fcb626feb4ac203 in term 1.
I20260812 03:03:23.773092 8708 leader_election.cc:302] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1638c83050fa44fc8fcb626feb4ac203, 263ca1115c074b4b9414fb436a041eb3; no voters:
I20260812 03:03:23.773350 9023 raft_consensus.cc:2781] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:23.773443 9023 raft_consensus.cc:686] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203 [term 1 LEADER]: Becoming Leader. State: Replica: 1638c83050fa44fc8fcb626feb4ac203, State: Running, Role: LEADER
I20260812 03:03:23.773577 9023 consensus_queue.cc:237] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } }
I20260812 03:03:23.774911 8140 catalog_manager.cc:5495] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1638c83050fa44fc8fcb626feb4ac203 (127.5.96.131). New cstate: current_term: 1 leader_uuid: "1638c83050fa44fc8fcb626feb4ac203" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1638c83050fa44fc8fcb626feb4ac203" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45757 } health_report { overall_health: HEALTHY } } }
I20260812 03:03:23.800675 8601 raft_consensus.cc:1261] T e3335a53796c445db8285ea8c7e3b76f P 263ca1115c074b4b9414fb436a041eb3 [term 1 FOLLOWER]: Refusing update from remote peer 1638c83050fa44fc8fcb626feb4ac203: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:23.800731 8349 raft_consensus.cc:1261] T e3335a53796c445db8285ea8c7e3b76f P f0d3c2ba5dbf4158879e784e90767ffd [term 1 FOLLOWER]: Refusing update from remote peer 1638c83050fa44fc8fcb626feb4ac203: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:23.801026 9023 consensus_queue.cc:1035] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203 [LEADER]: Connected to new peer: Peer: permanent_uuid: "263ca1115c074b4b9414fb436a041eb3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 38621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:23.801138 9051 consensus_queue.cc:1035] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0d3c2ba5dbf4158879e784e90767ffd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 03:03:23.812819 9030 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 03:03:25.358181 8140 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:25.358323 8140 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:25.360483 8140 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1fdf03fd2d9248a9a9e0c1e02a414010: Sending DeleteTablet for 3 replicas of tablet e3335a53796c445db8285ea8c7e3b76f
I20260812 03:03:25.361025 8775 tablet_service.cc:1514] Processing DeleteTablet for tablet e3335a53796c445db8285ea8c7e3b76f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:25 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:44642
I20260812 03:03:25.361032 8345 tablet_service.cc:1514] Processing DeleteTablet for tablet e3335a53796c445db8285ea8c7e3b76f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:25 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:43504
I20260812 03:03:25.361342 9138 tablet_replica.cc:330] T e3335a53796c445db8285ea8c7e3b76f P f0d3c2ba5dbf4158879e784e90767ffd: stopping tablet replica
I20260812 03:03:25.361336 8574 tablet_service.cc:1514] Processing DeleteTablet for tablet e3335a53796c445db8285ea8c7e3b76f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:25 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:35356
I20260812 03:03:25.361362 9139 tablet_replica.cc:330] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203: stopping tablet replica
I20260812 03:03:25.361430 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 8247
I20260812 03:03:25.361482 9139 raft_consensus.cc:2229] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:03:25.362501 9141 tablet_replica.cc:330] T e3335a53796c445db8285ea8c7e3b76f P 263ca1115c074b4b9414fb436a041eb3: stopping tablet replica
I20260812 03:03:25.362591 9141 raft_consensus.cc:2229] T e3335a53796c445db8285ea8c7e3b76f P 263ca1115c074b4b9414fb436a041eb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:25.362663 9141 raft_consensus.cc:2258] T e3335a53796c445db8285ea8c7e3b76f P 263ca1115c074b4b9414fb436a041eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:25.363171 9141 ts_tablet_manager.cc:1905] T e3335a53796c445db8285ea8c7e3b76f P 263ca1115c074b4b9414fb436a041eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:25.366276 9141 ts_tablet_manager.cc:1918] T e3335a53796c445db8285ea8c7e3b76f P 263ca1115c074b4b9414fb436a041eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:03:25.366338 9141 log.cc:1196] T e3335a53796c445db8285ea8c7e3b76f P 263ca1115c074b4b9414fb436a041eb3: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-1/wal/wals/e3335a53796c445db8285ea8c7e3b76f
I20260812 03:03:25.366539 9141 ts_tablet_manager.cc:1939] T e3335a53796c445db8285ea8c7e3b76f P 263ca1115c074b4b9414fb436a041eb3: Deleting consensus metadata
I20260812 03:03:25.367055 8122 catalog_manager.cc:4841] TS 263ca1115c074b4b9414fb436a041eb3 (127.5.96.130:38621): tablet e3335a53796c445db8285ea8c7e3b76f (table test-table [id=f6c45cb8be004387902972d2eeb600fc]) successfully deleted
I20260812 03:03:25.369755 9139 raft_consensus.cc:2258] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:25.370287 9139 ts_tablet_manager.cc:1905] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:25.372319 9139 ts_tablet_manager.cc:1918] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:03:25.372387 9139 log.cc:1196] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786503790637001-5506-0/minicluster-data/ts-2/wal/wals/e3335a53796c445db8285ea8c7e3b76f
I20260812 03:03:25.372599 9139 ts_tablet_manager.cc:1939] T e3335a53796c445db8285ea8c7e3b76f P 1638c83050fa44fc8fcb626feb4ac203: Deleting consensus metadata
I20260812 03:03:25.373173 8119 catalog_manager.cc:4841] TS 1638c83050fa44fc8fcb626feb4ac203 (127.5.96.131:45757): tablet e3335a53796c445db8285ea8c7e3b76f (table test-table [id=f6c45cb8be004387902972d2eeb600fc]) successfully deleted
W20260812 03:03:25.376430 8122 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.96.129:39297 (error 108)
I20260812 03:03:25.377139 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 8466
I20260812 03:03:25.377498 9146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:25.377065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:39297 (local address 127.0.0.1:43528)
0812 03:03:25.377329 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:25.377404 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.129:39297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":93,"threads_started":1}
W20260812 03:03:25.378866 8122 catalog_manager.cc:4584] TS f0d3c2ba5dbf4158879e784e90767ffd (127.5.96.129:39297): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e3335a53796c445db8285ea8c7e3b76f: Network error: Client connection negotiation failed: client connection to 127.5.96.129:39297: connect: Connection refused (error 111)
I20260812 03:03:25.386021 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 8677
I20260812 03:03:25.401472 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 8101
2026-08-12T03:03:25Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6875 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:03:25 dist-test-slave-6bbx krb5kdc[9149](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:03:25 dist-test-slave-6bbx krb5kdc[9149](info): set up 2 sockets
Aug 12 03:03:25 dist-test-slave-6bbx krb5kdc[9149](info): commencing operation
krb5kdc: starting...
W20260812 03:03:26.554752 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:03:26 dist-test-slave-6bbx krb5kdc[9149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:03:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:03:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:03:26.664379 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:42667
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:45713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:42667
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:26.772876 9175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:26.773092 9175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:26.773137 9175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:26.776935 9175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:03:26.776975 9175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:26.776993 9175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:26.777007 9175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:03:26.777025 9175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:03:26.781999 9175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:45713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:42667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:42667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:26.782846 9175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:26.783757 9175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:26.791898 9192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:26.791955 9175 server_base.cc:1034] running on GCE node
W20260812 03:03:26.791911 9190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:26.791908 9187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:26.792662 9175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:26.793637 9175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:26.794780 9175 hybrid_clock.cc:648] HybridClock initialized: now 1786503806794765 us; error 31 us; skew 500 ppm
Aug 12 03:03:26 dist-test-slave-6bbx krb5kdc[9149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503806, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:26.797287 9175 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:03:26.798059 9175 webserver.cc:460] Webserver started at http://127.5.96.190:43549/ using document root <none> and password file <none>
I20260812 03:03:26.798550 9175 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:26.798610 9175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:26.798799 9175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:26.800722 9175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "8f3bb8b37d69418ca68a5a52e8b8b264"
format_stamp: "Formatted at 2026-08-12 03:03:26 on dist-test-slave-6bbx"
server_key: "3392914e6ce9cfc10c7fc268e7854714"
server_key_iv: "85f2f105a55dcb2fc3d97da2cd96e079"
server_key_version: "encryptionkey@0"
I20260812 03:03:26.801251 9175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "8f3bb8b37d69418ca68a5a52e8b8b264"
format_stamp: "Formatted at 2026-08-12 03:03:26 on dist-test-slave-6bbx"
server_key: "3392914e6ce9cfc10c7fc268e7854714"
server_key_iv: "85f2f105a55dcb2fc3d97da2cd96e079"
server_key_version: "encryptionkey@0"
I20260812 03:03:26.804811 9175 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:03:26.806988 9204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:26.808132 9175 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:03:26.808248 9175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "8f3bb8b37d69418ca68a5a52e8b8b264"
format_stamp: "Formatted at 2026-08-12 03:03:26 on dist-test-slave-6bbx"
server_key: "3392914e6ce9cfc10c7fc268e7854714"
server_key_iv: "85f2f105a55dcb2fc3d97da2cd96e079"
server_key_version: "encryptionkey@0"
I20260812 03:03:26.808334 9175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:26.829746 9175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:26.832293 9175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:26.832458 9175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:26.839335 9175 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:42667
I20260812 03:03:26.839373 9289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:42667 every 8 connection(s)
I20260812 03:03:26.840245 9175 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:03:26.842927 9291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:26.848086 9291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f3bb8b37d69418ca68a5a52e8b8b264: Bootstrap starting.
I20260812 03:03:26.849607 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 9175
I20260812 03:03:26.849709 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:03:26.849932 5506 external_mini_cluster.cc:1428] Setting key 19b8bb6446c3e5eb2655e842cdaf6d3e
I20260812 03:03:26.850445 9291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f3bb8b37d69418ca68a5a52e8b8b264: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:26.851297 9291 log.cc:824] T 00000000000000000000000000000000 P 8f3bb8b37d69418ca68a5a52e8b8b264: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:26.853195 9291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f3bb8b37d69418ca68a5a52e8b8b264: No bootstrap required, opened a new log
I20260812 03:03:26.857093 9291 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f3bb8b37d69418ca68a5a52e8b8b264 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3bb8b37d69418ca68a5a52e8b8b264" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42667 } }
I20260812 03:03:26.857357 9291 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f3bb8b37d69418ca68a5a52e8b8b264 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:26.857417 9291 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f3bb8b37d69418ca68a5a52e8b8b264 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3bb8b37d69418ca68a5a52e8b8b264, State: Initialized, Role: FOLLOWER
I20260812 03:03:26.857894 9291 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f3bb8b37d69418ca68a5a52e8b8b264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3bb8b37d69418ca68a5a52e8b8b264" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42667 } }
I20260812 03:03:26.858023 9291 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f3bb8b37d69418ca68a5a52e8b8b264 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:03:26.858083 9291 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f3bb8b37d69418ca68a5a52e8b8b264 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:03:26.858174 9291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f3bb8b37d69418ca68a5a52e8b8b264 [term 0 FOLLOWER]: Advancing to term 1
Aug 12 03:03:26 dist-test-slave-6bbx krb5kdc[9149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:26.858985 9291 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f3bb8b37d69418ca68a5a52e8b8b264 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3bb8b37d69418ca68a5a52e8b8b264" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42667 } }
I20260812 03:03:26.859311 9291 leader_election.cc:302] T 00000000000000000000000000000000 P 8f3bb8b37d69418ca68a5a52e8b8b264 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f3bb8b37d69418ca68a5a52e8b8b264; no voters:
I20260812 03:03:26.859643 9291 leader_election.cc:288] T 00000000000000000000000000000000 P 8f3bb8b37d69418ca68a5a52e8b8b264 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:03:26.859756 9298 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f3bb8b37d69418ca68a5a52e8b8b264 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:26.860024 9298 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f3bb8b37d69418ca68a5a52e8b8b264 [term 1 LEADER]: Becoming Leader. State: Replica: 8f3bb8b37d69418ca68a5a52e8b8b264, State: Running, Role: LEADER
I20260812 03:03:26.860374 9298 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f3bb8b37d69418ca68a5a52e8b8b264 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3bb8b37d69418ca68a5a52e8b8b264" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42667 } }
I20260812 03:03:26.860924 9291 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f3bb8b37d69418ca68a5a52e8b8b264 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:03:26.861481 9298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f3bb8b37d69418ca68a5a52e8b8b264 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f3bb8b37d69418ca68a5a52e8b8b264. Latest consensus state: current_term: 1 leader_uuid: "8f3bb8b37d69418ca68a5a52e8b8b264" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3bb8b37d69418ca68a5a52e8b8b264" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42667 } } }
I20260812 03:03:26.862283 9298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f3bb8b37d69418ca68a5a52e8b8b264 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:26.862146 9300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f3bb8b37d69418ca68a5a52e8b8b264 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f3bb8b37d69418ca68a5a52e8b8b264" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3bb8b37d69418ca68a5a52e8b8b264" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42667 } } }
I20260812 03:03:26.862680 9300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f3bb8b37d69418ca68a5a52e8b8b264 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:26.863445 9310 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:03:26.864436 9295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:26.850932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55320 (local address 127.5.96.190:42667)
0812 03:03:26.851315 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:26.851330 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:26.851364 (+ 34us) server_negotiation.cc:407] Connection header received
0812 03:03:26.852124 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:26.852150 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:26.852450 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:26.852709 (+ 259us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:26.853574 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:26.854527 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:26.855707 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:26.856155 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:26.856268 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:26.858928 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:26.858933 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:26.858945 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:26.858970 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:26.860796 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:26.861267 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:26.861273 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:26.861278 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:26.861426 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:26.861762 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:26.861765 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:26.861767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:26.862274 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:26.863029 (+ 755us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:26.863181 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 03:03:26.863485 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":104,"threads_started":1}
I20260812 03:03:26.866174 9310 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:03:26.870929 9310 catalog_manager.cc:1347] Generated new cluster ID: 3d326b5ad4474656a6e8d2c50ade275c
I20260812 03:03:26.870990 9310 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:03:26.876681 9310 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:03:26.877504 9310 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:03:26.887635 9310 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f3bb8b37d69418ca68a5a52e8b8b264: Generated new TSK 0
I20260812 03:03:26.888195 9310 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.96.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
I20260812 03:03:26.917882 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:0
--local_ip_for_outbound_sockets=127.5.96.129
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:42667
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:45713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:27.060283 9325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:27.060496 9325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:27.060534 9325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:27.064621 9325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:27.064675 9325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:27.064730 9325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:03:27.069620 9325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:45713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:42667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:27.070590 9325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:27.071792 9325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:27.081624 9337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:27.081637 9332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:27.081724 9335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:27.081815 9325 server_base.cc:1034] running on GCE node
I20260812 03:03:27.082243 9325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:27.082803 9325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:27.083977 9325 hybrid_clock.cc:648] HybridClock initialized: now 1786503807083962 us; error 30 us; skew 500 ppm
Aug 12 03:03:27 dist-test-slave-6bbx krb5kdc[9149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503807, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:27.086503 9325 init.cc:377] Logged in from keytab as kudu/127.5.96.129@KRBTEST.COM (short username kudu)
I20260812 03:03:27.087272 9325 webserver.cc:460] Webserver started at http://127.5.96.129:35665/ using document root <none> and password file <none>
I20260812 03:03:27.087802 9325 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:27.087873 9325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:27.088061 9325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:27.089952 9325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/data/instance:
uuid: "508cbe10f5874b30962be5cf172a9d2c"
format_stamp: "Formatted at 2026-08-12 03:03:27 on dist-test-slave-6bbx"
server_key: "320d2ed1d26297253b803eef58173576"
server_key_iv: "53bc0c67ce010fe332828d8e299c4138"
server_key_version: "encryptionkey@0"
I20260812 03:03:27.090454 9325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance:
uuid: "508cbe10f5874b30962be5cf172a9d2c"
format_stamp: "Formatted at 2026-08-12 03:03:27 on dist-test-slave-6bbx"
server_key: "320d2ed1d26297253b803eef58173576"
server_key_iv: "53bc0c67ce010fe332828d8e299c4138"
server_key_version: "encryptionkey@0"
I20260812 03:03:27.093870 9325 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260812 03:03:27.096052 9352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:27.097134 9325 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:03:27.097239 9325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "508cbe10f5874b30962be5cf172a9d2c"
format_stamp: "Formatted at 2026-08-12 03:03:27 on dist-test-slave-6bbx"
server_key: "320d2ed1d26297253b803eef58173576"
server_key_iv: "53bc0c67ce010fe332828d8e299c4138"
server_key_version: "encryptionkey@0"
I20260812 03:03:27.097325 9325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:27.115504 9325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:27.117864 9325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:27.118014 9325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:27.118503 9325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:27.119256 9325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:27.119302 9325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:27.119351 9325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:27.119380 9325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:27.127362 9325 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:41367
I20260812 03:03:27.127390 9523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:41367 every 8 connection(s)
I20260812 03:03:27.128145 9325 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:03:27.133605 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 9325
I20260812 03:03:27.133702 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance
I20260812 03:03:27.133955 5506 external_mini_cluster.cc:1428] Setting key 182704fbf848bd0f11aa14c5723d1f5c
Aug 12 03:03:27 dist-test-slave-6bbx krb5kdc[9149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503807, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:27.141090 9295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:27.129714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:56737 (local address 127.5.96.190:42667)
0812 03:03:27.129826 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:27.129830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:27.130557 (+ 727us) server_negotiation.cc:407] Connection header received
0812 03:03:27.131828 (+ 1271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:27.131831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:27.131890 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:27.131965 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:27.133059 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:27.133589 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:27.134816 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:27.135089 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:27.135184 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:27.137732 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:27.137736 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:27.137738 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:27.137766 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:27.139072 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:27.139606 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:27.139609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:27.139610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:27.139722 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:27.140061 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:27.140063 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:27.140065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:27.140228 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:27.140298 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:27.140870 (+ 572us) server_negotiation.cc:299] Negotiation successful
0812 03:03:27.140971 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 03:03:27.141878 9526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:27.130004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:42667 (local address 127.5.96.129:56737)
0812 03:03:27.130420 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:27.130459 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:27.131598 (+ 1139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:27.132140 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:27.132147 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:27.132486 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:27.132916 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:27.132928 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:27.133712 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:27.133715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:27.134707 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:27.134717 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:27.135226 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:27.135230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:27.135402 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:27.135980 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:27.136000 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:27.137573 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:27.139190 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:27.139195 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:27.139209 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:27.139522 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:27.139799 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:27.139802 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:27.139804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:27.139975 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:27.140328 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:27.140335 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:27.140622 (+ 287us) client_negotiation.cc:769] Sending connection context
0812 03:03:27.140834 (+ 212us) client_negotiation.cc:240] Negotiation successful
0812 03:03:27.141141 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":98,"threads_started":1}
I20260812 03:03:27.142975 9524 heartbeater.cc:347] Connected to a master server at 127.5.96.190:42667
I20260812 03:03:27.143172 9524 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:27.143577 9524 heartbeater.cc:511] Master 127.5.96.190:42667 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.5.96.130@KRBTEST.COM; defaulting to no policy
I20260812 03:03:27.145046 9240 ts_manager.cc:194] Registered new tserver with Master: 508cbe10f5874b30962be5cf172a9d2c (127.5.96.129:41367)
I20260812 03:03:27.146155 9240 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.129@KRBTEST.COM'} at 127.5.96.129:56737
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
I20260812 03:03:27.159523 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:0
--local_ip_for_outbound_sockets=127.5.96.130
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:42667
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:45713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:27.264535 9535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:27.264755 9535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:27.264809 9535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:27.268616 9535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:27.268663 9535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:27.268738 9535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:03:27.273590 9535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:45713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:42667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:27.274464 9535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:27.275352 9535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:27.284181 9544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:27.284188 9542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:27.284344 9546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:27.284379 9535 server_base.cc:1034] running on GCE node
I20260812 03:03:27.284833 9535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:27.285357 9535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:27.286505 9535 hybrid_clock.cc:648] HybridClock initialized: now 1786503807286491 us; error 25 us; skew 500 ppm
Aug 12 03:03:27 dist-test-slave-6bbx krb5kdc[9149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503807, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:27.289116 9535 init.cc:377] Logged in from keytab as kudu/127.5.96.130@KRBTEST.COM (short username kudu)
I20260812 03:03:27.289872 9535 webserver.cc:460] Webserver started at http://127.5.96.130:46827/ using document root <none> and password file <none>
I20260812 03:03:27.290360 9535 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:27.290424 9535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:27.290601 9535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:27.292476 9535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/data/instance:
uuid: "d626df98317b45c893f4a7abafb8c78b"
format_stamp: "Formatted at 2026-08-12 03:03:27 on dist-test-slave-6bbx"
server_key: "1356f32b33863891aa4acd15306caf37"
server_key_iv: "abf0acbdb894512bc498d33d77d94dc4"
server_key_version: "encryptionkey@0"
I20260812 03:03:27.292989 9535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance:
uuid: "d626df98317b45c893f4a7abafb8c78b"
format_stamp: "Formatted at 2026-08-12 03:03:27 on dist-test-slave-6bbx"
server_key: "1356f32b33863891aa4acd15306caf37"
server_key_iv: "abf0acbdb894512bc498d33d77d94dc4"
server_key_version: "encryptionkey@0"
I20260812 03:03:27.296351 9535 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 03:03:27.298432 9554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:27.299490 9535 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:27.299593 9535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "d626df98317b45c893f4a7abafb8c78b"
format_stamp: "Formatted at 2026-08-12 03:03:27 on dist-test-slave-6bbx"
server_key: "1356f32b33863891aa4acd15306caf37"
server_key_iv: "abf0acbdb894512bc498d33d77d94dc4"
server_key_version: "encryptionkey@0"
I20260812 03:03:27.299697 9535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:27.311542 9535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:27.314081 9535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:27.314239 9535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:27.314743 9535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:27.315472 9535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:27.315518 9535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:27.315567 9535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:27.315594 9535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:27.323493 9535 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:38001
I20260812 03:03:27.323544 9723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:38001 every 8 connection(s)
I20260812 03:03:27.324285 9535 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:03:27.324472 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 9535
I20260812 03:03:27.324565 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance
I20260812 03:03:27.324844 5506 external_mini_cluster.cc:1428] Setting key 397cd90119ac12bb8060e73f1a46851d
Aug 12 03:03:27 dist-test-slave-6bbx krb5kdc[9149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503807, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.96.131@KRBTEST.COM; defaulting to no policy
I20260812 03:03:27.337795 9295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:27.326056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:50383 (local address 127.5.96.190:42667)
0812 03:03:27.326169 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:27.326173 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:27.327064 (+ 891us) server_negotiation.cc:407] Connection header received
0812 03:03:27.328423 (+ 1359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:27.328427 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:27.328484 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:27.328559 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:27.329933 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:27.330646 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:27.331803 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:27.332139 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:27.332252 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:27.334664 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:27.334668 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:27.334672 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:27.334701 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:27.336080 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:27.336545 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:27.336548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:27.336549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:27.336624 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:27.336871 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:27.336873 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:27.336874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:27.337026 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:27.337094 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:27.337542 (+ 448us) server_negotiation.cc:299] Negotiation successful
0812 03:03:27.337638 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.131@KRBTEST.COM" created.
I20260812 03:03:27.338608 9727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:27.326473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:42667 (local address 127.5.96.130:50383)
0812 03:03:27.326909 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:27.326957 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:27.328201 (+ 1244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:27.328737 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:27.328753 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:27.329184 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:27.329790 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:27.329806 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:27.330773 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:27.330776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:27.331660 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:27.331701 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:27.332279 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:27.332282 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:27.332413 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:27.332934 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:27.332949 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:27.334484 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:27.336183 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:27.336188 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:27.336198 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:27.336471 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:27.336691 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:27.336693 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:27.336695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:27.336814 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:27.337092 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:27.337096 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:27.337361 (+ 265us) client_negotiation.cc:769] Sending connection context
0812 03:03:27.337504 (+ 143us) client_negotiation.cc:240] Negotiation successful
0812 03:03:27.337790 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":99,"threads_started":1}
I20260812 03:03:27.339702 9725 heartbeater.cc:347] Connected to a master server at 127.5.96.190:42667
I20260812 03:03:27.339916 9725 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:27.340361 9725 heartbeater.cc:511] Master 127.5.96.190:42667 requested a full tablet report, sending...
I20260812 03:03:27.341372 9240 ts_manager.cc:194] Registered new tserver with Master: d626df98317b45c893f4a7abafb8c78b (127.5.96.130:38001)
I20260812 03:03:27.341939 9240 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.130@KRBTEST.COM'} at 127.5.96.130:50383
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
I20260812 03:03:27.350904 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:0
--local_ip_for_outbound_sockets=127.5.96.131
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:42667
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:45713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:27.460246 9730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:27.460479 9730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:27.460532 9730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:27.464478 9730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:27.464525 9730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:27.464598 9730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:03:27.469519 9730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:45713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:42667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:27.470413 9730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:27.471318 9730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:27.480327 9741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:27.480322 9743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:27.480660 9730 server_base.cc:1034] running on GCE node
W20260812 03:03:27.480333 9739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:27.481315 9730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:27.481835 9730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:27.482990 9730 hybrid_clock.cc:648] HybridClock initialized: now 1786503807482959 us; error 47 us; skew 500 ppm
Aug 12 03:03:27 dist-test-slave-6bbx krb5kdc[9149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503807, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:27.485488 9730 init.cc:377] Logged in from keytab as kudu/127.5.96.131@KRBTEST.COM (short username kudu)
I20260812 03:03:27.486300 9730 webserver.cc:460] Webserver started at http://127.5.96.131:41011/ using document root <none> and password file <none>
I20260812 03:03:27.486804 9730 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:27.486868 9730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:27.487046 9730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:27.488940 9730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/data/instance:
uuid: "86b82448dd6e4834a93f73a00be03ec1"
format_stamp: "Formatted at 2026-08-12 03:03:27 on dist-test-slave-6bbx"
server_key: "c9ef356be9b7d8f30522765487bbb18a"
server_key_iv: "dc84ea64912226129de3664fd03ef186"
server_key_version: "encryptionkey@0"
I20260812 03:03:27.489451 9730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance:
uuid: "86b82448dd6e4834a93f73a00be03ec1"
format_stamp: "Formatted at 2026-08-12 03:03:27 on dist-test-slave-6bbx"
server_key: "c9ef356be9b7d8f30522765487bbb18a"
server_key_iv: "dc84ea64912226129de3664fd03ef186"
server_key_version: "encryptionkey@0"
I20260812 03:03:27.492983 9730 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260812 03:03:27.495141 9758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:27.496273 9730 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 03:03:27.496376 9730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "86b82448dd6e4834a93f73a00be03ec1"
format_stamp: "Formatted at 2026-08-12 03:03:27 on dist-test-slave-6bbx"
server_key: "c9ef356be9b7d8f30522765487bbb18a"
server_key_iv: "dc84ea64912226129de3664fd03ef186"
server_key_version: "encryptionkey@0"
I20260812 03:03:27.496459 9730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:27.520006 9730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:27.522375 9730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:27.522532 9730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:27.523047 9730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:27.523829 9730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:27.523876 9730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:27.523934 9730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:27.523962 9730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:27.532173 9730 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:42811
I20260812 03:03:27.532199 9927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:42811 every 8 connection(s)
I20260812 03:03:27.532953 9730 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:03:27.536369 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 9730
I20260812 03:03:27.536476 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance
I20260812 03:03:27.536784 5506 external_mini_cluster.cc:1428] Setting key e3c51f41c39df2d92f085c7ead919ba0
Aug 12 03:03:27 dist-test-slave-6bbx krb5kdc[9149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503807, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:27.545073 9295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:27.534513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:46115 (local address 127.5.96.190:42667)
0812 03:03:27.534628 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:27.534631 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:27.535355 (+ 724us) server_negotiation.cc:407] Connection header received
0812 03:03:27.536763 (+ 1408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:27.536766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:27.536818 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:27.536904 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:27.538125 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:27.538706 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:27.539811 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:27.540037 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:27.540127 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:27.542312 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:27.542316 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:27.542318 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:27.542341 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:27.543383 (+ 1042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:27.543846 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:27.543848 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:27.543850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:27.543919 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:27.544171 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:27.544173 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:27.544174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:27.544329 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:27.544397 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:27.544869 (+ 472us) server_negotiation.cc:299] Negotiation successful
0812 03:03:27.544957 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:03:27.545759 9930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:27.534800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:42667 (local address 127.5.96.131:46115)
0812 03:03:27.535207 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:27.535255 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:27.536528 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:27.537106 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:27.537113 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:27.537478 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:27.538000 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:27.538016 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:27.538822 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:27.538826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:27.539703 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:27.539714 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:27.540176 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:27.540180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:27.540360 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:27.540805 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:27.540820 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:27.542162 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:27.543476 (+ 1314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:27.543481 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:27.543491 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:27.543774 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:27.543982 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:27.543984 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:27.543986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:27.544105 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:27.544397 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:27.544403 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:27.544680 (+ 277us) client_negotiation.cc:769] Sending connection context
0812 03:03:27.544827 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 03:03:27.545047 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":105,"threads_started":1}
I20260812 03:03:27.546715 9928 heartbeater.cc:347] Connected to a master server at 127.5.96.190:42667
I20260812 03:03:27.546903 9928 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:27.547302 9928 heartbeater.cc:511] Master 127.5.96.190:42667 requested a full tablet report, sending...
I20260812 03:03:27.548187 9240 ts_manager.cc:194] Registered new tserver with Master: 86b82448dd6e4834a93f73a00be03ec1 (127.5.96.131:42811)
I20260812 03:03:27.548687 9240 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.131@KRBTEST.COM'} at 127.5.96.131:46115
I20260812 03:03:27.550170 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 03:03:27.563592 9295 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:27.562407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55326 (local address 127.5.96.190:42667)
0812 03:03:27.562548 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:27.562551 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:27.562713 (+ 162us) server_negotiation.cc:407] Connection header received
0812 03:03:27.562821 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:27.562823 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:27.562859 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:27.562913 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:27.563500 (+ 587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55326: BlockingRecv error: recv got EOF from 127.0.0.1:55326 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:03:27.564610 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 9325
I20260812 03:03:27.579018 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 9535
I20260812 03:03:27.594244 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 9730
I20260812 03:03:27.611105 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 9175
2026-08-12T03:03:27Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2209 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:03:27 dist-test-slave-6bbx krb5kdc[9941](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:03:27 dist-test-slave-6bbx krb5kdc[9941](info): set up 2 sockets
Aug 12 03:03:27 dist-test-slave-6bbx krb5kdc[9941](info): commencing operation
krb5kdc: starting...
W20260812 03:03:28.783208 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:03:28 dist-test-slave-6bbx krb5kdc[9941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:03:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:03:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:03:28.892490 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:34659
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:34659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:28.998404 9956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:28.998631 9956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:28.998670 9956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:29.002696 9956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:03:29.002751 9956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:29.002770 9956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:29.002800 9956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:03:29.002826 9956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:03:29.007846 9956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:34659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:34659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:29.008773 9956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:29.009702 9956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:29.017328 9972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:29.017343 9967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:29.017339 9969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:29.017540 9956 server_base.cc:1034] running on GCE node
I20260812 03:03:29.018134 9956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:29.019124 9956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:29.020272 9956 hybrid_clock.cc:648] HybridClock initialized: now 1786503809020246 us; error 36 us; skew 500 ppm
Aug 12 03:03:29 dist-test-slave-6bbx krb5kdc[9941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503809, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:29.025480 9956 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:03:29.026279 9956 webserver.cc:460] Webserver started at http://127.5.96.190:43957/ using document root <none> and password file <none>
I20260812 03:03:29.026772 9956 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:29.026831 9956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:29.027019 9956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:29.028913 9956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "76d8236616cf4273934d6cee8b43e975"
format_stamp: "Formatted at 2026-08-12 03:03:29 on dist-test-slave-6bbx"
server_key: "0a062765a2494d9366ae67e7dfe77c5a"
server_key_iv: "658dfc1462945862f74916cba2b13eb8"
server_key_version: "encryptionkey@0"
I20260812 03:03:29.029425 9956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "76d8236616cf4273934d6cee8b43e975"
format_stamp: "Formatted at 2026-08-12 03:03:29 on dist-test-slave-6bbx"
server_key: "0a062765a2494d9366ae67e7dfe77c5a"
server_key_iv: "658dfc1462945862f74916cba2b13eb8"
server_key_version: "encryptionkey@0"
I20260812 03:03:29.032799 9956 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:03:29.034883 9980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:29.036010 9956 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:03:29.036118 9956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "76d8236616cf4273934d6cee8b43e975"
format_stamp: "Formatted at 2026-08-12 03:03:29 on dist-test-slave-6bbx"
server_key: "0a062765a2494d9366ae67e7dfe77c5a"
server_key_iv: "658dfc1462945862f74916cba2b13eb8"
server_key_version: "encryptionkey@0"
I20260812 03:03:29.036201 9956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:29.059958 9956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:29.062361 9956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:29.062536 9956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:29.069137 9956 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:34659
I20260812 03:03:29.069496 10069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:34659 every 8 connection(s)
I20260812 03:03:29.069998 9956 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:03:29.072595 10070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:29.077345 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 9956
I20260812 03:03:29.077450 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:03:29.077677 5506 external_mini_cluster.cc:1428] Setting key 202c0d4f886367b94c844dcdf5cd5670
I20260812 03:03:29.080277 10070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76d8236616cf4273934d6cee8b43e975: Bootstrap starting.
I20260812 03:03:29.083169 10070 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76d8236616cf4273934d6cee8b43e975: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:29.084075 10070 log.cc:824] T 00000000000000000000000000000000 P 76d8236616cf4273934d6cee8b43e975: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:29.088747 10070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76d8236616cf4273934d6cee8b43e975: No bootstrap required, opened a new log
Aug 12 03:03:29 dist-test-slave-6bbx krb5kdc[9941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:29.092326 10070 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76d8236616cf4273934d6cee8b43e975 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d8236616cf4273934d6cee8b43e975" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 34659 } }
I20260812 03:03:29.092590 10070 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76d8236616cf4273934d6cee8b43e975 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:29.092661 10070 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76d8236616cf4273934d6cee8b43e975 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76d8236616cf4273934d6cee8b43e975, State: Initialized, Role: FOLLOWER
I20260812 03:03:29.093120 10070 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76d8236616cf4273934d6cee8b43e975 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d8236616cf4273934d6cee8b43e975" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 34659 } }
I20260812 03:03:29.093258 10070 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76d8236616cf4273934d6cee8b43e975 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:03:29.093312 10070 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76d8236616cf4273934d6cee8b43e975 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:03:29.093410 10070 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76d8236616cf4273934d6cee8b43e975 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:29.094182 10070 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76d8236616cf4273934d6cee8b43e975 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d8236616cf4273934d6cee8b43e975" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 34659 } }
I20260812 03:03:29.094496 10070 leader_election.cc:302] T 00000000000000000000000000000000 P 76d8236616cf4273934d6cee8b43e975 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76d8236616cf4273934d6cee8b43e975; no voters:
I20260812 03:03:29.094766 10070 leader_election.cc:288] T 00000000000000000000000000000000 P 76d8236616cf4273934d6cee8b43e975 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:03:29.094866 10080 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76d8236616cf4273934d6cee8b43e975 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:29.095109 10080 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76d8236616cf4273934d6cee8b43e975 [term 1 LEADER]: Becoming Leader. State: Replica: 76d8236616cf4273934d6cee8b43e975, State: Running, Role: LEADER
I20260812 03:03:29.095463 10080 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76d8236616cf4273934d6cee8b43e975 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d8236616cf4273934d6cee8b43e975" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 34659 } }
I20260812 03:03:29.096164 10070 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76d8236616cf4273934d6cee8b43e975 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:03:29.096987 10082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76d8236616cf4273934d6cee8b43e975 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76d8236616cf4273934d6cee8b43e975. Latest consensus state: current_term: 1 leader_uuid: "76d8236616cf4273934d6cee8b43e975" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d8236616cf4273934d6cee8b43e975" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 34659 } } }
I20260812 03:03:29.097030 10083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76d8236616cf4273934d6cee8b43e975 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76d8236616cf4273934d6cee8b43e975" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d8236616cf4273934d6cee8b43e975" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 34659 } } }
I20260812 03:03:29.097213 10083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76d8236616cf4273934d6cee8b43e975 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:29.097211 10082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76d8236616cf4273934d6cee8b43e975 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:29.097942 10078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:29.083594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60408 (local address 127.5.96.190:34659)
0812 03:03:29.084578 (+ 984us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:29.084601 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:29.084641 (+ 40us) server_negotiation.cc:407] Connection header received
0812 03:03:29.085397 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:29.085419 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:29.085796 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:29.087021 (+ 1225us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:29.087437 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:29.088928 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:29.089831 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:29.090309 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:29.090462 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:29.093057 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:29.093062 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:29.093074 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:29.093099 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:29.094727 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:29.095235 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:29.095241 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:29.095247 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:29.095384 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:29.095749 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:29.095752 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:29.095754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:29.096231 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:29.096430 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:29.096696 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 03:03:29.096994 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":832,"thread_start_us":109,"threads_started":1}
I20260812 03:03:29.100773 5506 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 03:03:29.101061 10100 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 76d8236616cf4273934d6cee8b43e975: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:03:29.101140 10100 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:03:29.101215 10091 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:03:29.102057 10091 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:03:29.107445 10091 catalog_manager.cc:1347] Generated new cluster ID: 336074518c5b4eeca94e67ad80e5f5ee
I20260812 03:03:29.107555 10091 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:03:29.115584 10091 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:03:29.116669 10091 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:03:29.119059 10078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:29.114927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60420 (local address 127.5.96.190:34659)
0812 03:03:29.115016 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:29.115020 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:29.115062 (+ 42us) server_negotiation.cc:407] Connection header received
0812 03:03:29.115185 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:29.115188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:29.115231 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:29.115298 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:29.115912 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:29.116402 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:29.117220 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:29.117468 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:29.117553 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:29.118259 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:29.118262 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:29.118264 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:03:29.118626 (+ 362us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:29.118651 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:03:29.118666 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:29.118741 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:29.118851 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 03:03:29.118923 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":36}
I20260812 03:03:29.130403 10091 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76d8236616cf4273934d6cee8b43e975: Generated new TSK 0
I20260812 03:03:29.131031 10091 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:03:29.135710 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 9956
2026-08-12T03:03:29Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1522 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:03:29 dist-test-slave-6bbx krb5kdc[10118](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:03:29 dist-test-slave-6bbx krb5kdc[10118](info): set up 2 sockets
Aug 12 03:03:29 dist-test-slave-6bbx krb5kdc[10118](info): commencing operation
krb5kdc: starting...
W20260812 03:03:30.287248 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:03:30 dist-test-slave-6bbx krb5kdc[10118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:03:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:03:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:03:30.398061 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:35921
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:40029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:35921
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:30.505805 10137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:30.506026 10137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:30.506078 10137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:30.510010 10137 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:03:30.510057 10137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:30.510083 10137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:30.510106 10137 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:03:30.510133 10137 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:03:30.515072 10137 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:35921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:35921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:30.515964 10137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:30.516851 10137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:03:30.524688 10137 server_base.cc:1034] running on GCE node
W20260812 03:03:30.524655 10145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:30.524667 10146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:30.524657 10148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:30.525444 10137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:30.526396 10137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:30.527549 10137 hybrid_clock.cc:648] HybridClock initialized: now 1786503810527538 us; error 24 us; skew 500 ppm
Aug 12 03:03:30 dist-test-slave-6bbx krb5kdc[10118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503810, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:30.530063 10137 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:03:30.530881 10137 webserver.cc:460] Webserver started at http://127.5.96.190:45661/ using document root <none> and password file <none>
I20260812 03:03:30.531361 10137 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:30.531419 10137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:30.531603 10137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:30.533456 10137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "26862ae5a092471d99706e90a10a6b03"
format_stamp: "Formatted at 2026-08-12 03:03:30 on dist-test-slave-6bbx"
server_key: "561f63607bbe3cd31f92909156c4cac4"
server_key_iv: "b9b19e3ed1bd6a56bf78eda910c1b972"
server_key_version: "encryptionkey@0"
I20260812 03:03:30.533967 10137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "26862ae5a092471d99706e90a10a6b03"
format_stamp: "Formatted at 2026-08-12 03:03:30 on dist-test-slave-6bbx"
server_key: "561f63607bbe3cd31f92909156c4cac4"
server_key_iv: "b9b19e3ed1bd6a56bf78eda910c1b972"
server_key_version: "encryptionkey@0"
I20260812 03:03:30.537310 10137 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260812 03:03:30.539383 10164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:30.540474 10137 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:30.540586 10137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "26862ae5a092471d99706e90a10a6b03"
format_stamp: "Formatted at 2026-08-12 03:03:30 on dist-test-slave-6bbx"
server_key: "561f63607bbe3cd31f92909156c4cac4"
server_key_iv: "b9b19e3ed1bd6a56bf78eda910c1b972"
server_key_version: "encryptionkey@0"
I20260812 03:03:30.540674 10137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:30.549894 10137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:30.552330 10137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:30.552484 10137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:30.558893 10137 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:35921
I20260812 03:03:30.559137 10246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:35921 every 8 connection(s)
I20260812 03:03:30.559736 10137 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:03:30.562270 10249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:30.562758 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 10137
I20260812 03:03:30.562871 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:03:30.563153 5506 external_mini_cluster.cc:1428] Setting key 7c35494a519416f935b8babb7ceee0ee
I20260812 03:03:30.568763 10249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26862ae5a092471d99706e90a10a6b03: Bootstrap starting.
I20260812 03:03:30.571273 10249 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26862ae5a092471d99706e90a10a6b03: Neither blocks nor log segments found. Creating new log.
Aug 12 03:03:30 dist-test-slave-6bbx krb5kdc[10118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:30.572079 10249 log.cc:824] T 00000000000000000000000000000000 P 26862ae5a092471d99706e90a10a6b03: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:30.573835 10249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26862ae5a092471d99706e90a10a6b03: No bootstrap required, opened a new log
I20260812 03:03:30.576362 10252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:30.564388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41458 (local address 127.5.96.190:35921)
0812 03:03:30.564763 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:30.564778 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:30.564812 (+ 34us) server_negotiation.cc:407] Connection header received
0812 03:03:30.565458 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:30.565473 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:30.565770 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:30.566021 (+ 251us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:30.566884 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:30.567896 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:30.569031 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:30.569472 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:30.569581 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:30.572256 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:30.572261 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:30.572273 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:30.572299 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:30.573769 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:30.574235 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:30.574240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:30.574244 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:30.574355 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:30.574620 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:30.574622 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:30.574623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:30.574971 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:30.575102 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:30.575390 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 03:03:30.575642 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":99,"threads_started":1}
I20260812 03:03:30.577561 10249 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26862ae5a092471d99706e90a10a6b03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26862ae5a092471d99706e90a10a6b03" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35921 } }
I20260812 03:03:30.577764 10249 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26862ae5a092471d99706e90a10a6b03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:30.577809 10249 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26862ae5a092471d99706e90a10a6b03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26862ae5a092471d99706e90a10a6b03, State: Initialized, Role: FOLLOWER
I20260812 03:03:30.578192 10249 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26862ae5a092471d99706e90a10a6b03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26862ae5a092471d99706e90a10a6b03" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35921 } }
I20260812 03:03:30.578328 10249 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26862ae5a092471d99706e90a10a6b03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:03:30.578428 10249 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26862ae5a092471d99706e90a10a6b03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:03:30.578518 10249 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26862ae5a092471d99706e90a10a6b03 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:30.579336 10249 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26862ae5a092471d99706e90a10a6b03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26862ae5a092471d99706e90a10a6b03" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35921 } }
I20260812 03:03:30.579641 10249 leader_election.cc:302] T 00000000000000000000000000000000 P 26862ae5a092471d99706e90a10a6b03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26862ae5a092471d99706e90a10a6b03; no voters:
I20260812 03:03:30.579946 10249 leader_election.cc:288] T 00000000000000000000000000000000 P 26862ae5a092471d99706e90a10a6b03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:03:30.580039 10254 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26862ae5a092471d99706e90a10a6b03 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:30.580240 10254 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26862ae5a092471d99706e90a10a6b03 [term 1 LEADER]: Becoming Leader. State: Replica: 26862ae5a092471d99706e90a10a6b03, State: Running, Role: LEADER
I20260812 03:03:30.580567 10254 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26862ae5a092471d99706e90a10a6b03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26862ae5a092471d99706e90a10a6b03" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35921 } }
I20260812 03:03:30.581058 10249 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26862ae5a092471d99706e90a10a6b03 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:03:30.582064 10257 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26862ae5a092471d99706e90a10a6b03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26862ae5a092471d99706e90a10a6b03. Latest consensus state: current_term: 1 leader_uuid: "26862ae5a092471d99706e90a10a6b03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26862ae5a092471d99706e90a10a6b03" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35921 } } }
I20260812 03:03:30.582206 10257 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26862ae5a092471d99706e90a10a6b03 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:30.582525 10256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26862ae5a092471d99706e90a10a6b03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26862ae5a092471d99706e90a10a6b03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26862ae5a092471d99706e90a10a6b03" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35921 } } }
I20260812 03:03:30.582590 10271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:03:30.582685 10256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26862ae5a092471d99706e90a10a6b03 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:30.585680 10271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:03:30.591033 10271 catalog_manager.cc:1347] Generated new cluster ID: 8c4be4b5434942cfb17e48ab301268c5
I20260812 03:03:30.591082 10271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:03:30.600203 10271 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:03:30.600996 10271 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:03:30.610966 10271 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26862ae5a092471d99706e90a10a6b03: Generated new TSK 0
I20260812 03:03:30.611511 10271 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.96.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
I20260812 03:03:30.639428 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:0
--local_ip_for_outbound_sockets=127.5.96.129
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:35921
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:40029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:30.746546 10288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:30.746783 10288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:30.746841 10288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:30.750703 10288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:30.750757 10288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:30.750833 10288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:03:30.755774 10288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:35921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:30.756677 10288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:30.757591 10288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:30.766460 10299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:30.766523 10297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:30.766655 10288 server_base.cc:1034] running on GCE node
W20260812 03:03:30.766523 10296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:30.767097 10288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:30.767693 10288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:30.768877 10288 hybrid_clock.cc:648] HybridClock initialized: now 1786503810768860 us; error 28 us; skew 500 ppm
Aug 12 03:03:30 dist-test-slave-6bbx krb5kdc[10118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503810, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:30.771437 10288 init.cc:377] Logged in from keytab as kudu/127.5.96.129@KRBTEST.COM (short username kudu)
I20260812 03:03:30.772246 10288 webserver.cc:460] Webserver started at http://127.5.96.129:34485/ using document root <none> and password file <none>
I20260812 03:03:30.772725 10288 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:30.772782 10288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:30.772940 10288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:30.774749 10288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/data/instance:
uuid: "9ddf2c656e6d470898e4cfec9775ca3d"
format_stamp: "Formatted at 2026-08-12 03:03:30 on dist-test-slave-6bbx"
server_key: "8d229b5689d07c9e17055a5f9c3fd85e"
server_key_iv: "b6070ce48d83020181d64acf8b407d59"
server_key_version: "encryptionkey@0"
I20260812 03:03:30.775241 10288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance:
uuid: "9ddf2c656e6d470898e4cfec9775ca3d"
format_stamp: "Formatted at 2026-08-12 03:03:30 on dist-test-slave-6bbx"
server_key: "8d229b5689d07c9e17055a5f9c3fd85e"
server_key_iv: "b6070ce48d83020181d64acf8b407d59"
server_key_version: "encryptionkey@0"
I20260812 03:03:30.778620 10288 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 03:03:30.780844 10310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:30.781966 10288 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:03:30.782063 10288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "9ddf2c656e6d470898e4cfec9775ca3d"
format_stamp: "Formatted at 2026-08-12 03:03:30 on dist-test-slave-6bbx"
server_key: "8d229b5689d07c9e17055a5f9c3fd85e"
server_key_iv: "b6070ce48d83020181d64acf8b407d59"
server_key_version: "encryptionkey@0"
I20260812 03:03:30.782174 10288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:30.801513 10288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:30.804366 10288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:30.804504 10288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:30.805027 10288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:30.805745 10288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:30.805801 10288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:30.805837 10288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:30.805857 10288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:30.814030 10288 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:44297
I20260812 03:03:30.814069 10487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:44297 every 8 connection(s)
I20260812 03:03:30.814855 10288 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
Aug 12 03:03:30 dist-test-slave-6bbx krb5kdc[10118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503810, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:30.825017 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 10288
I20260812 03:03:30.825133 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance
I20260812 03:03:30.825454 5506 external_mini_cluster.cc:1428] Setting key a708b17ca3fa56b43d2f7075b615f274
I20260812 03:03:30.827304 10252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:30.816503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:40375 (local address 127.5.96.190:35921)
0812 03:03:30.816668 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:30.816672 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:30.817446 (+ 774us) server_negotiation.cc:407] Connection header received
0812 03:03:30.818728 (+ 1282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:30.818731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:30.818776 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:30.818848 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:30.819951 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:30.820478 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:30.821531 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:30.821751 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:30.821815 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:30.824015 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:30.824018 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:30.824020 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:30.824045 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:30.825165 (+ 1120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:30.825730 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:30.825733 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:30.825734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:30.825810 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:30.826182 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:30.826185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:30.826187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:30.826410 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:30.826507 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:30.826993 (+ 486us) server_negotiation.cc:299] Negotiation successful
0812 03:03:30.827144 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 03:03:30.828061 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:30.816844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:35921 (local address 127.5.96.129:40375)
0812 03:03:30.817293 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:30.817342 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:30.818548 (+ 1206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:30.818986 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:30.818992 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:30.819341 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:30.819829 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:30.819844 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:30.820592 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:30.820595 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:30.821436 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:30.821446 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:30.821850 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:30.821852 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:30.821980 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:30.822447 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:30.822462 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:30.823866 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:30.825299 (+ 1433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:30.825305 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:30.825318 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:30.825642 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:30.825900 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:30.825904 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:30.825906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:30.826079 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:30.826501 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:30.826505 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:30.826767 (+ 262us) client_negotiation.cc:769] Sending connection context
0812 03:03:30.826935 (+ 168us) client_negotiation.cc:240] Negotiation successful
0812 03:03:30.827227 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":119,"threads_started":1}
I20260812 03:03:30.829164 10488 heartbeater.cc:347] Connected to a master server at 127.5.96.190:35921
I20260812 03:03:30.829375 10488 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:30.829798 10488 heartbeater.cc:511] Master 127.5.96.190:35921 requested a full tablet report, sending...
I20260812 03:03:30.831300 10194 ts_manager.cc:194] Registered new tserver with Master: 9ddf2c656e6d470898e4cfec9775ca3d (127.5.96.129:44297)
I20260812 03:03:30.832856 10194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.129@KRBTEST.COM'} at 127.5.96.129:40375
WARNING: no policy specified for kudu/127.5.96.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
I20260812 03:03:30.850943 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:0
--local_ip_for_outbound_sockets=127.5.96.130
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:35921
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:40029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:30.957260 10497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:30.957491 10497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:30.957546 10497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:30.961464 10497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:30.961516 10497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:30.961591 10497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:03:30.966742 10497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:35921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:30.967684 10497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:30.968604 10497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:30.977470 10509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:30.977550 10497 server_base.cc:1034] running on GCE node
W20260812 03:03:30.977464 10513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:30.977502 10505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:30.978071 10497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:30.978616 10497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:30.979725 10497 hybrid_clock.cc:648] HybridClock initialized: now 1786503810979700 us; error 35 us; skew 500 ppm
Aug 12 03:03:30 dist-test-slave-6bbx krb5kdc[10118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503810, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:30.982352 10497 init.cc:377] Logged in from keytab as kudu/127.5.96.130@KRBTEST.COM (short username kudu)
I20260812 03:03:30.983153 10497 webserver.cc:460] Webserver started at http://127.5.96.130:46509/ using document root <none> and password file <none>
I20260812 03:03:30.983635 10497 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:30.983731 10497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:30.983938 10497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:30.986040 10497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/data/instance:
uuid: "950615cc0bab46839fd18a679023a644"
format_stamp: "Formatted at 2026-08-12 03:03:30 on dist-test-slave-6bbx"
server_key: "6ab30bf0d8d624809491a20f4d7d3b89"
server_key_iv: "21aa99177089ca58ef69a49ec33aa7ac"
server_key_version: "encryptionkey@0"
I20260812 03:03:30.986722 10497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance:
uuid: "950615cc0bab46839fd18a679023a644"
format_stamp: "Formatted at 2026-08-12 03:03:30 on dist-test-slave-6bbx"
server_key: "6ab30bf0d8d624809491a20f4d7d3b89"
server_key_iv: "21aa99177089ca58ef69a49ec33aa7ac"
server_key_version: "encryptionkey@0"
I20260812 03:03:30.990321 10497 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260812 03:03:30.992584 10521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:30.993757 10497 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:03:30.993863 10497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "950615cc0bab46839fd18a679023a644"
format_stamp: "Formatted at 2026-08-12 03:03:30 on dist-test-slave-6bbx"
server_key: "6ab30bf0d8d624809491a20f4d7d3b89"
server_key_iv: "21aa99177089ca58ef69a49ec33aa7ac"
server_key_version: "encryptionkey@0"
I20260812 03:03:30.993952 10497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:31.019268 10497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:31.021833 10497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:31.022001 10497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:31.022562 10497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:31.023337 10497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:31.023386 10497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:31.023437 10497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:31.023466 10497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:31.031491 10497 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:41275
I20260812 03:03:31.031519 10681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:41275 every 8 connection(s)
I20260812 03:03:31.032311 10497 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:03:31.036199 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 10497
I20260812 03:03:31.036322 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance
I20260812 03:03:31.036605 5506 external_mini_cluster.cc:1428] Setting key 409921daf2fc0eaabebb8825675711a3
Aug 12 03:03:31 dist-test-slave-6bbx krb5kdc[10118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503810, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:31.045941 10252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:31.033909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:48289 (local address 127.5.96.190:35921)
0812 03:03:31.034064 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:31.034069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:31.034820 (+ 751us) server_negotiation.cc:407] Connection header received
0812 03:03:31.036148 (+ 1328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:31.036152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:31.036213 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:31.036309 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:31.037535 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:31.038182 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:31.039361 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:31.039688 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:31.039791 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:31.042382 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:31.042385 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:31.042387 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:31.042410 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:31.043833 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:31.044430 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:31.044433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:31.044434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:31.044511 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:31.044882 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:31.044884 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:31.044886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:31.045073 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:31.045176 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:31.045724 (+ 548us) server_negotiation.cc:299] Negotiation successful
0812 03:03:31.045821 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:03:31.047035 10685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:31.034256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:35921 (local address 127.5.96.130:48289)
0812 03:03:31.034678 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:31.034721 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:31.035928 (+ 1207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:31.036495 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:31.036502 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:31.036880 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:31.037378 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:31.037399 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:31.038300 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:31.038303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:31.039238 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:31.039249 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:31.039818 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:31.039822 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:31.040017 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:31.040543 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:31.040562 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:31.042222 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:31.043948 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:31.043955 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:31.043970 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:31.044339 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:31.044598 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:31.044601 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:31.044603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:31.044795 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:31.045185 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:31.045191 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:31.045491 (+ 300us) client_negotiation.cc:769] Sending connection context
0812 03:03:31.045698 (+ 207us) client_negotiation.cc:240] Negotiation successful
0812 03:03:31.046030 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":113,"threads_started":1}
WARNING: no policy specified for kudu/127.5.96.131@KRBTEST.COM; defaulting to no policy
I20260812 03:03:31.048146 10682 heartbeater.cc:347] Connected to a master server at 127.5.96.190:35921
I20260812 03:03:31.048373 10682 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:31.048859 10682 heartbeater.cc:511] Master 127.5.96.190:35921 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.131@KRBTEST.COM" created.
I20260812 03:03:31.049875 10194 ts_manager.cc:194] Registered new tserver with Master: 950615cc0bab46839fd18a679023a644 (127.5.96.130:41275)
I20260812 03:03:31.050556 10194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.130@KRBTEST.COM'} at 127.5.96.130:48289
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
I20260812 03:03:31.062361 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:0
--local_ip_for_outbound_sockets=127.5.96.131
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:35921
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:40029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:31.172351 10688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:31.172573 10688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:31.172628 10688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:31.176456 10688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:31.176505 10688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:31.176575 10688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:03:31.181507 10688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:35921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:31.182448 10688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:31.183411 10688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:31.192145 10699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:31.192118 10695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:31.192131 10694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:31.192322 10688 server_base.cc:1034] running on GCE node
I20260812 03:03:31.192827 10688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:31.193397 10688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:31.194554 10688 hybrid_clock.cc:648] HybridClock initialized: now 1786503811194537 us; error 28 us; skew 500 ppm
Aug 12 03:03:31 dist-test-slave-6bbx krb5kdc[10118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503811, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:31.197108 10688 init.cc:377] Logged in from keytab as kudu/127.5.96.131@KRBTEST.COM (short username kudu)
I20260812 03:03:31.197932 10688 webserver.cc:460] Webserver started at http://127.5.96.131:41301/ using document root <none> and password file <none>
I20260812 03:03:31.198421 10688 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:31.198484 10688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:31.198679 10688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:31.200582 10688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/data/instance:
uuid: "5eae30b5ed424aee93dceea6933a4e81"
format_stamp: "Formatted at 2026-08-12 03:03:31 on dist-test-slave-6bbx"
server_key: "f4f0a353db3e67bb557cc8b5ff8d0fb6"
server_key_iv: "8b68e2cce6f238257414a26710f01b74"
server_key_version: "encryptionkey@0"
I20260812 03:03:31.201102 10688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance:
uuid: "5eae30b5ed424aee93dceea6933a4e81"
format_stamp: "Formatted at 2026-08-12 03:03:31 on dist-test-slave-6bbx"
server_key: "f4f0a353db3e67bb557cc8b5ff8d0fb6"
server_key_iv: "8b68e2cce6f238257414a26710f01b74"
server_key_version: "encryptionkey@0"
I20260812 03:03:31.204483 10688 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:03:31.206575 10710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:31.207646 10688 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:31.207785 10688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "5eae30b5ed424aee93dceea6933a4e81"
format_stamp: "Formatted at 2026-08-12 03:03:31 on dist-test-slave-6bbx"
server_key: "f4f0a353db3e67bb557cc8b5ff8d0fb6"
server_key_iv: "8b68e2cce6f238257414a26710f01b74"
server_key_version: "encryptionkey@0"
I20260812 03:03:31.207871 10688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:31.219731 10688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:31.222067 10688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:31.222236 10688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:31.222745 10688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:31.223474 10688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:31.223520 10688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:31.223569 10688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:31.223598 10688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:31.231693 10688 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:46605
I20260812 03:03:31.231707 10880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:46605 every 8 connection(s)
I20260812 03:03:31.232466 10688 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:03:31.237502 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 10688
I20260812 03:03:31.237599 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance
I20260812 03:03:31.237840 5506 external_mini_cluster.cc:1428] Setting key deda8979f1144d917f56e29fd5a7259c
Aug 12 03:03:31 dist-test-slave-6bbx krb5kdc[10118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503811, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:31.244853 10252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:31.234048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:44711 (local address 127.5.96.190:35921)
0812 03:03:31.234177 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:31.234180 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:31.234896 (+ 716us) server_negotiation.cc:407] Connection header received
0812 03:03:31.236185 (+ 1289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:31.236188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:31.236255 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:31.236326 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:31.237380 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:31.237950 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:31.239345 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:31.239567 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:31.239698 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:31.241876 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:31.241879 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:31.241882 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:31.241906 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:31.243131 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:31.243564 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:31.243567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:31.243568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:31.243636 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:31.243956 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:31.243959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:31.243960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:31.244127 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:31.244218 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:31.244641 (+ 423us) server_negotiation.cc:299] Negotiation successful
0812 03:03:31.244738 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 03:03:31.245514 10883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:31.234345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:35921 (local address 127.5.96.131:44711)
0812 03:03:31.234759 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:31.234798 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:31.235974 (+ 1176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:31.236491 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:31.236498 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:31.236825 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:31.237242 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:31.237254 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:31.238061 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:31.238064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:31.239236 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:31.239247 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:31.239759 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:31.239763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:31.239934 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:31.240381 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:31.240396 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:31.241720 (+ 1324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:31.243241 (+ 1521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:31.243246 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:31.243259 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:31.243493 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:31.243742 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:31.243744 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:31.243746 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:31.243875 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:31.244201 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:31.244206 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:31.244454 (+ 248us) client_negotiation.cc:769] Sending connection context
0812 03:03:31.244594 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 03:03:31.244833 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":115,"threads_started":1}
I20260812 03:03:31.246424 10881 heartbeater.cc:347] Connected to a master server at 127.5.96.190:35921
I20260812 03:03:31.246604 10881 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:31.246996 10881 heartbeater.cc:511] Master 127.5.96.190:35921 requested a full tablet report, sending...
I20260812 03:03:31.247893 10194 ts_manager.cc:194] Registered new tserver with Master: 5eae30b5ed424aee93dceea6933a4e81 (127.5.96.131:46605)
I20260812 03:03:31.248402 10194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.131@KRBTEST.COM'} at 127.5.96.131:44711
I20260812 03:03:31.251152 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 03:03:31 dist-test-slave-6bbx krb5kdc[10118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503811, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 03:03:31 dist-test-slave-6bbx krb5kdc[10118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503811, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:31.275485 10252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:31.268211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41472 (local address 127.5.96.190:35921)
0812 03:03:31.268318 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:31.268322 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:31.268522 (+ 200us) server_negotiation.cc:407] Connection header received
0812 03:03:31.268639 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:31.268642 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:31.268680 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:31.268744 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:31.269409 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:31.269895 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:31.270687 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:31.270894 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:31.270979 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:31.273130 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:31.273135 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:31.273137 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:31.273163 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:31.274218 (+ 1055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:31.274549 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:31.274551 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:31.274552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:31.274618 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:31.274797 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:31.274799 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:31.274800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:31.274985 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:31.275083 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:31.275251 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 03:03:31.275338 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260812 03:03:31.275883 10194 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41472
I20260812 03:03:31.276695 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 10288
I20260812 03:03:31.290192 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 10497
I20260812 03:03:31.301697 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 10688
I20260812 03:03:31.310191 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 10137
2026-08-12T03:03:31Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2177 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:03:31 dist-test-slave-6bbx krb5kdc[10902](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:03:31 dist-test-slave-6bbx krb5kdc[10902](info): set up 2 sockets
Aug 12 03:03:31 dist-test-slave-6bbx krb5kdc[10902](info): commencing operation
krb5kdc: starting...
W20260812 03:03:32.466871 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:03:32 dist-test-slave-6bbx krb5kdc[10902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:03:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:03:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:03:32.575724 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:45859
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:46283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:45859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:32.681051 10924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:32.681284 10924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:32.681320 10924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:32.685168 10924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:03:32.685214 10924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:32.685231 10924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:32.685249 10924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:03:32.685266 10924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:03:32.690397 10924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:46283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:45859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:45859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:32.691272 10924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:32.692265 10924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:32.699957 10932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:32.700019 10936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:32.700079 10924 server_base.cc:1034] running on GCE node
W20260812 03:03:32.699957 10931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:32.700668 10924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:32.701673 10924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:32.702834 10924 hybrid_clock.cc:648] HybridClock initialized: now 1786503812702818 us; error 26 us; skew 500 ppm
Aug 12 03:03:32 dist-test-slave-6bbx krb5kdc[10902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503812, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:32.705476 10924 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:03:32.706264 10924 webserver.cc:460] Webserver started at http://127.5.96.190:33743/ using document root <none> and password file <none>
I20260812 03:03:32.706756 10924 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:32.706823 10924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:32.707005 10924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:32.708899 10924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "36bf64feb12945a9bc79b18d62277d30"
format_stamp: "Formatted at 2026-08-12 03:03:32 on dist-test-slave-6bbx"
server_key: "61df5c9cdee5e85947e16884a41fa965"
server_key_iv: "56de9cf57a9ce2f82a0f08785f01d0cb"
server_key_version: "encryptionkey@0"
I20260812 03:03:32.709420 10924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "36bf64feb12945a9bc79b18d62277d30"
format_stamp: "Formatted at 2026-08-12 03:03:32 on dist-test-slave-6bbx"
server_key: "61df5c9cdee5e85947e16884a41fa965"
server_key_iv: "56de9cf57a9ce2f82a0f08785f01d0cb"
server_key_version: "encryptionkey@0"
I20260812 03:03:32.712847 10924 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:03:32.715070 10944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:32.716192 10924 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:03:32.716316 10924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "36bf64feb12945a9bc79b18d62277d30"
format_stamp: "Formatted at 2026-08-12 03:03:32 on dist-test-slave-6bbx"
server_key: "61df5c9cdee5e85947e16884a41fa965"
server_key_iv: "56de9cf57a9ce2f82a0f08785f01d0cb"
server_key_version: "encryptionkey@0"
I20260812 03:03:32.716401 10924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:32.736967 10924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:32.739825 10924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:32.739982 10924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:32.746487 10924 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:45859
I20260812 03:03:32.746501 11021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:45859 every 8 connection(s)
I20260812 03:03:32.747344 10924 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:03:32.749850 11022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:32.750567 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 10924
I20260812 03:03:32.750697 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:03:32.750936 5506 external_mini_cluster.cc:1428] Setting key 4bf576b6f4cfc2736dcb42ae8e35834f
I20260812 03:03:32.755897 11022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36bf64feb12945a9bc79b18d62277d30: Bootstrap starting.
I20260812 03:03:32.758219 11022 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36bf64feb12945a9bc79b18d62277d30: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:32.758868 11022 log.cc:824] T 00000000000000000000000000000000 P 36bf64feb12945a9bc79b18d62277d30: Log is configured to *not* fsync() on all Append() calls
Aug 12 03:03:32 dist-test-slave-6bbx krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:32.760814 11022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36bf64feb12945a9bc79b18d62277d30: No bootstrap required, opened a new log
I20260812 03:03:32.764945 11027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:32.752207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51442 (local address 127.5.96.190:45859)
0812 03:03:32.752585 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:32.752604 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:32.752645 (+ 41us) server_negotiation.cc:407] Connection header received
0812 03:03:32.753434 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:32.753457 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:32.753878 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:32.754231 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:32.754823 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:32.756251 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:32.757076 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:32.757738 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:32.757890 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:32.760221 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:32.760227 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:32.760242 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:32.760271 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:32.762040 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:32.762465 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:32.762472 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:32.762477 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:32.762617 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:32.762841 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:32.762844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:32.762847 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:32.763343 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:32.763520 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:32.763767 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 03:03:32.764054 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":99,"threads_started":1}
I20260812 03:03:32.765053 11022 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36bf64feb12945a9bc79b18d62277d30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36bf64feb12945a9bc79b18d62277d30" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45859 } }
I20260812 03:03:32.765292 11022 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36bf64feb12945a9bc79b18d62277d30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:32.765349 11022 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36bf64feb12945a9bc79b18d62277d30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36bf64feb12945a9bc79b18d62277d30, State: Initialized, Role: FOLLOWER
I20260812 03:03:32.765790 11022 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36bf64feb12945a9bc79b18d62277d30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36bf64feb12945a9bc79b18d62277d30" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45859 } }
I20260812 03:03:32.765936 11022 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36bf64feb12945a9bc79b18d62277d30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:03:32.766002 11022 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36bf64feb12945a9bc79b18d62277d30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:03:32.766091 11022 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36bf64feb12945a9bc79b18d62277d30 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:32.766880 11022 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36bf64feb12945a9bc79b18d62277d30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36bf64feb12945a9bc79b18d62277d30" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45859 } }
I20260812 03:03:32.767202 11022 leader_election.cc:302] T 00000000000000000000000000000000 P 36bf64feb12945a9bc79b18d62277d30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36bf64feb12945a9bc79b18d62277d30; no voters:
I20260812 03:03:32.767457 11022 leader_election.cc:288] T 00000000000000000000000000000000 P 36bf64feb12945a9bc79b18d62277d30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:03:32.767556 11029 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36bf64feb12945a9bc79b18d62277d30 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:32.767830 11029 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36bf64feb12945a9bc79b18d62277d30 [term 1 LEADER]: Becoming Leader. State: Replica: 36bf64feb12945a9bc79b18d62277d30, State: Running, Role: LEADER
I20260812 03:03:32.768105 11029 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36bf64feb12945a9bc79b18d62277d30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36bf64feb12945a9bc79b18d62277d30" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45859 } }
I20260812 03:03:32.768524 11022 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36bf64feb12945a9bc79b18d62277d30 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:03:32.769647 11033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36bf64feb12945a9bc79b18d62277d30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36bf64feb12945a9bc79b18d62277d30. Latest consensus state: current_term: 1 leader_uuid: "36bf64feb12945a9bc79b18d62277d30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36bf64feb12945a9bc79b18d62277d30" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45859 } } }
I20260812 03:03:32.769793 11033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36bf64feb12945a9bc79b18d62277d30 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:32.769647 11032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36bf64feb12945a9bc79b18d62277d30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36bf64feb12945a9bc79b18d62277d30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36bf64feb12945a9bc79b18d62277d30" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45859 } } }
I20260812 03:03:32.770051 11032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36bf64feb12945a9bc79b18d62277d30 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:32.770243 11042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:03:32.773474 11042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:03:32.778791 11042 catalog_manager.cc:1347] Generated new cluster ID: 846355164f764651a01dc3aacc7663fc
I20260812 03:03:32.778855 11042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:03:32.797744 11042 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:03:32.798739 11042 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:03:32.803510 11042 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36bf64feb12945a9bc79b18d62277d30: Generated new TSK 0
I20260812 03:03:32.804106 11042 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.96.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
I20260812 03:03:32.834203 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:0
--local_ip_for_outbound_sockets=127.5.96.129
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:45859
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:46283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:32.958640 11064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:32.958854 11064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:32.958892 11064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:32.962826 11064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:32.962875 11064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:32.962929 11064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:03:32.967834 11064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:46283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:45859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:32.968686 11064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:32.969636 11064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:32.978585 11076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:32.978633 11071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:32.978629 11073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:32.978838 11064 server_base.cc:1034] running on GCE node
I20260812 03:03:32.979193 11064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:32.979799 11064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:32.980957 11064 hybrid_clock.cc:648] HybridClock initialized: now 1786503812980942 us; error 26 us; skew 500 ppm
Aug 12 03:03:32 dist-test-slave-6bbx krb5kdc[10902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503812, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:32.983578 11064 init.cc:377] Logged in from keytab as kudu/127.5.96.129@KRBTEST.COM (short username kudu)
I20260812 03:03:32.984385 11064 webserver.cc:460] Webserver started at http://127.5.96.129:39647/ using document root <none> and password file <none>
I20260812 03:03:32.984884 11064 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:32.984948 11064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:32.985131 11064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:32.987166 11064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-0/data/instance:
uuid: "92e4993eb9f24147924bae1f487e9458"
format_stamp: "Formatted at 2026-08-12 03:03:32 on dist-test-slave-6bbx"
server_key: "1bc5990f412738cbbb7b8d8184522ad2"
server_key_iv: "7b3d1e12066028f58cec90b318db3658"
server_key_version: "encryptionkey@0"
I20260812 03:03:32.987748 11064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance:
uuid: "92e4993eb9f24147924bae1f487e9458"
format_stamp: "Formatted at 2026-08-12 03:03:32 on dist-test-slave-6bbx"
server_key: "1bc5990f412738cbbb7b8d8184522ad2"
server_key_iv: "7b3d1e12066028f58cec90b318db3658"
server_key_version: "encryptionkey@0"
I20260812 03:03:32.991230 11064 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:03:32.993515 11088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:32.994650 11064 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:32.994758 11064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "92e4993eb9f24147924bae1f487e9458"
format_stamp: "Formatted at 2026-08-12 03:03:32 on dist-test-slave-6bbx"
server_key: "1bc5990f412738cbbb7b8d8184522ad2"
server_key_iv: "7b3d1e12066028f58cec90b318db3658"
server_key_version: "encryptionkey@0"
I20260812 03:03:32.994846 11064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:33.004812 11064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:33.007717 11064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:33.007881 11064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:33.008430 11064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:33.009195 11064 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:33.009244 11064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:33.009295 11064 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:33.009325 11064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:33.017570 11064 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:38237
I20260812 03:03:33.017618 11260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:38237 every 8 connection(s)
I20260812 03:03:33.018435 11064 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:03:33.020061 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 11064
I20260812 03:03:33.020175 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance
I20260812 03:03:33.020478 5506 external_mini_cluster.cc:1428] Setting key 31efb3256b0d12e19151a7abae7800f8
Aug 12 03:03:33 dist-test-slave-6bbx krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503812, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.96.130@KRBTEST.COM; defaulting to no policy
I20260812 03:03:33.032672 11027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:33.020153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:59923 (local address 127.5.96.190:45859)
0812 03:03:33.020318 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:33.020323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:33.021197 (+ 874us) server_negotiation.cc:407] Connection header received
0812 03:03:33.022629 (+ 1432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:33.022632 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:33.022684 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:33.022755 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:33.024078 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:33.024943 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:33.026164 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:33.026507 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:33.026608 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:33.029347 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:33.029352 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:33.029354 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:33.029382 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:33.030703 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:33.031254 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:33.031257 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:33.031259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:33.031346 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:33.031702 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:33.031705 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:33.031706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:33.031882 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:33.031955 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:33.032435 (+ 480us) server_negotiation.cc:299] Negotiation successful
0812 03:03:33.032541 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.130@KRBTEST.COM" created.
I20260812 03:03:33.033500 11269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:33.020613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:45859 (local address 127.5.96.129:59923)
0812 03:03:33.021053 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:33.021107 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:33.022433 (+ 1326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:33.022920 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:33.022926 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:33.023252 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:33.023917 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:33.023938 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:33.025168 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:33.025171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:33.026035 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:33.026045 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:33.026671 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:33.026674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:33.026821 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:33.027397 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:33.027412 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:33.029145 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:33.030830 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:33.030837 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:33.030851 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:33.031164 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:33.031430 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:33.031434 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:33.031435 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:33.031608 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:33.031959 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:33.031964 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:33.032223 (+ 259us) client_negotiation.cc:769] Sending connection context
0812 03:03:33.032374 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 03:03:33.032605 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":126,"threads_started":1}
I20260812 03:03:33.034675 11265 heartbeater.cc:347] Connected to a master server at 127.5.96.190:45859
I20260812 03:03:33.034889 11265 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:33.035430 11265 heartbeater.cc:511] Master 127.5.96.190:45859 requested a full tablet report, sending...
I20260812 03:03:33.036903 10968 ts_manager.cc:194] Registered new tserver with Master: 92e4993eb9f24147924bae1f487e9458 (127.5.96.129:38237)
I20260812 03:03:33.038079 10968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.129@KRBTEST.COM'} at 127.5.96.129:59923
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
I20260812 03:03:33.045887 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:0
--local_ip_for_outbound_sockets=127.5.96.130
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:45859
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:46283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:33.155390 11273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:33.155612 11273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:33.155695 11273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:33.159570 11273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:33.159622 11273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:33.159716 11273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:03:33.164685 11273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:46283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:45859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:33.165601 11273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:33.166515 11273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:33.175503 11280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:33.175518 11281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:33.175740 11273 server_base.cc:1034] running on GCE node
W20260812 03:03:33.175520 11285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:33.176155 11273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:33.176741 11273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:33.177893 11273 hybrid_clock.cc:648] HybridClock initialized: now 1786503813177878 us; error 26 us; skew 500 ppm
Aug 12 03:03:33 dist-test-slave-6bbx krb5kdc[10902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503813, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:33.180464 11273 init.cc:377] Logged in from keytab as kudu/127.5.96.130@KRBTEST.COM (short username kudu)
I20260812 03:03:33.181272 11273 webserver.cc:460] Webserver started at http://127.5.96.130:34095/ using document root <none> and password file <none>
I20260812 03:03:33.181768 11273 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:33.181834 11273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:33.182013 11273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:33.183933 11273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-1/data/instance:
uuid: "6d45097876674662a645577549005f7c"
format_stamp: "Formatted at 2026-08-12 03:03:33 on dist-test-slave-6bbx"
server_key: "607ca9714fdb894eb70e76ff7c14079f"
server_key_iv: "6e62134fc2981c0b940a85df1a9dbcec"
server_key_version: "encryptionkey@0"
I20260812 03:03:33.184451 11273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance:
uuid: "6d45097876674662a645577549005f7c"
format_stamp: "Formatted at 2026-08-12 03:03:33 on dist-test-slave-6bbx"
server_key: "607ca9714fdb894eb70e76ff7c14079f"
server_key_iv: "6e62134fc2981c0b940a85df1a9dbcec"
server_key_version: "encryptionkey@0"
I20260812 03:03:33.187850 11273 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 03:03:33.190030 11295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:33.191136 11273 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:33.191251 11273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "6d45097876674662a645577549005f7c"
format_stamp: "Formatted at 2026-08-12 03:03:33 on dist-test-slave-6bbx"
server_key: "607ca9714fdb894eb70e76ff7c14079f"
server_key_iv: "6e62134fc2981c0b940a85df1a9dbcec"
server_key_version: "encryptionkey@0"
I20260812 03:03:33.191339 11273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:33.208586 11273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:33.210980 11273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:33.211141 11273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:33.211649 11273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:33.212426 11273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:33.212473 11273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:33.212522 11273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:33.212550 11273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:33.220623 11273 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:42327
I20260812 03:03:33.220664 11464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:42327 every 8 connection(s)
I20260812 03:03:33.221398 11273 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
Aug 12 03:03:33 dist-test-slave-6bbx krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503813, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:33.231068 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 11273
I20260812 03:03:33.231190 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance
I20260812 03:03:33.231487 5506 external_mini_cluster.cc:1428] Setting key 4a56835b65f1a3649d245cd5563e2db5
I20260812 03:03:33.233445 11027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:33.222939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:49385 (local address 127.5.96.190:45859)
0812 03:03:33.223062 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:33.223066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:33.223850 (+ 784us) server_negotiation.cc:407] Connection header received
0812 03:03:33.225089 (+ 1239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:33.225091 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:33.225136 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:33.225204 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:33.226194 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:33.226696 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:33.227764 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:33.227977 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:33.228051 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:33.230138 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:33.230141 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:33.230143 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:33.230167 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:33.231430 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:33.231993 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:33.231996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:33.231997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:33.232075 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:33.232415 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:33.232419 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:33.232420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:33.232586 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:33.232674 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:33.233228 (+ 554us) server_negotiation.cc:299] Negotiation successful
0812 03:03:33.233329 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:03:33.234308 11473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:33.223263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:45859 (local address 127.5.96.130:49385)
0812 03:03:33.223720 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:33.223763 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:33.224914 (+ 1151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:33.225340 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:33.225346 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:33.225652 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:33.226090 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:33.226101 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:33.226793 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:33.226795 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:33.227613 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:33.227621 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:33.228085 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:33.228088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:33.228214 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:33.228687 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:33.228717 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:33.229994 (+ 1277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:33.231531 (+ 1537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:33.231537 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:33.231550 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:33.231899 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:33.232171 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:33.232175 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:33.232176 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:33.232316 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:33.232690 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:33.232696 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:33.232959 (+ 263us) client_negotiation.cc:769] Sending connection context
0812 03:03:33.233149 (+ 190us) client_negotiation.cc:240] Negotiation successful
0812 03:03:33.233446 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":124,"threads_started":1}
I20260812 03:03:33.235601 11468 heartbeater.cc:347] Connected to a master server at 127.5.96.190:45859
I20260812 03:03:33.235821 11468 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:33.236255 11468 heartbeater.cc:511] Master 127.5.96.190:45859 requested a full tablet report, sending...
I20260812 03:03:33.237216 10968 ts_manager.cc:194] Registered new tserver with Master: 6d45097876674662a645577549005f7c (127.5.96.130:42327)
I20260812 03:03:33.237970 10968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.130@KRBTEST.COM'} at 127.5.96.130:49385
WARNING: no policy specified for kudu/127.5.96.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
I20260812 03:03:33.256600 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:0
--local_ip_for_outbound_sockets=127.5.96.131
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:45859
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:46283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:33.364240 11477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:33.364462 11477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:33.364501 11477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:33.368407 11477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:33.368453 11477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:33.368518 11477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:03:33.373732 11477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:46283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:45859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:33.374622 11477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:33.375545 11477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:33.384615 11486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:33.384631 11483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:33.384621 11484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:33.384671 11477 server_base.cc:1034] running on GCE node
I20260812 03:03:33.385231 11477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:33.385789 11477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:33.386938 11477 hybrid_clock.cc:648] HybridClock initialized: now 1786503813386923 us; error 26 us; skew 500 ppm
Aug 12 03:03:33 dist-test-slave-6bbx krb5kdc[10902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503813, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:33.389490 11477 init.cc:377] Logged in from keytab as kudu/127.5.96.131@KRBTEST.COM (short username kudu)
I20260812 03:03:33.390298 11477 webserver.cc:460] Webserver started at http://127.5.96.131:41469/ using document root <none> and password file <none>
I20260812 03:03:33.390805 11477 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:33.390863 11477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:33.391052 11477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:33.393064 11477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-2/data/instance:
uuid: "b35538a6ff1c4e288e60f23b8a1ad177"
format_stamp: "Formatted at 2026-08-12 03:03:33 on dist-test-slave-6bbx"
server_key: "958cc3ff7d2a32fa1f7a962f1f253014"
server_key_iv: "0f77df67d321c8b32c8d99a0d897f668"
server_key_version: "encryptionkey@0"
I20260812 03:03:33.393569 11477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance:
uuid: "b35538a6ff1c4e288e60f23b8a1ad177"
format_stamp: "Formatted at 2026-08-12 03:03:33 on dist-test-slave-6bbx"
server_key: "958cc3ff7d2a32fa1f7a962f1f253014"
server_key_iv: "0f77df67d321c8b32c8d99a0d897f668"
server_key_version: "encryptionkey@0"
I20260812 03:03:33.397076 11477 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:03:33.399353 11497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:33.400470 11477 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:03:33.400568 11477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "b35538a6ff1c4e288e60f23b8a1ad177"
format_stamp: "Formatted at 2026-08-12 03:03:33 on dist-test-slave-6bbx"
server_key: "958cc3ff7d2a32fa1f7a962f1f253014"
server_key_iv: "0f77df67d321c8b32c8d99a0d897f668"
server_key_version: "encryptionkey@0"
I20260812 03:03:33.400658 11477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:33.421811 11477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:33.424360 11477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:33.424517 11477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:33.425045 11477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:33.425810 11477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:33.425855 11477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:33.425895 11477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:33.425922 11477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:33.434077 11477 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:38509
I20260812 03:03:33.434103 11669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:38509 every 8 connection(s)
I20260812 03:03:33.434871 11477 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:03:33.442027 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 11477
I20260812 03:03:33.442142 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance
I20260812 03:03:33.442391 5506 external_mini_cluster.cc:1428] Setting key bfa6e9d5570018d03550bc05350f1a3e
Aug 12 03:03:33 dist-test-slave-6bbx krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503813, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:33.446842 11027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:33.436520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:55283 (local address 127.5.96.190:45859)
0812 03:03:33.436651 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:33.436655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:33.437430 (+ 775us) server_negotiation.cc:407] Connection header received
0812 03:03:33.438716 (+ 1286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:33.438718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:33.438764 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:33.438833 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:33.439890 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:33.440416 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:33.441430 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:33.441638 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:33.441708 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:33.444212 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:33.444215 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:33.444218 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:33.444254 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:33.445341 (+ 1087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:33.445751 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:33.445754 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:33.445755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:33.445818 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:33.446043 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:33.446045 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:33.446046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:33.446192 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:33.446271 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:33.446642 (+ 371us) server_negotiation.cc:299] Negotiation successful
0812 03:03:33.446728 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 03:03:33.447540 11675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:33.436861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:45859 (local address 127.5.96.131:55283)
0812 03:03:33.437280 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:33.437319 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:33.438538 (+ 1219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:33.438976 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:33.438982 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:33.439304 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:33.439776 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:33.439791 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:33.440505 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:33.440507 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:33.441342 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:33.441351 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:33.441738 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:33.441740 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:33.441864 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:33.442371 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:33.442386 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:33.444063 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:33.445435 (+ 1372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:33.445440 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:33.445450 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:33.445680 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:33.445869 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:33.445871 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:33.445872 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:33.445983 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:33.446270 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:33.446274 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:33.446471 (+ 197us) client_negotiation.cc:769] Sending connection context
0812 03:03:33.446609 (+ 138us) client_negotiation.cc:240] Negotiation successful
0812 03:03:33.446840 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":111,"threads_started":1}
I20260812 03:03:33.448484 11671 heartbeater.cc:347] Connected to a master server at 127.5.96.190:45859
I20260812 03:03:33.448685 11671 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:33.449100 11671 heartbeater.cc:511] Master 127.5.96.190:45859 requested a full tablet report, sending...
I20260812 03:03:33.449968 10968 ts_manager.cc:194] Registered new tserver with Master: b35538a6ff1c4e288e60f23b8a1ad177 (127.5.96.131:38509)
I20260812 03:03:33.450479 10968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.131@KRBTEST.COM'} at 127.5.96.131:55283
I20260812 03:03:33.455744 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 03:03:33 dist-test-slave-6bbx krb5kdc[10902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:03:33 dist-test-slave-6bbx krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.129@KRBTEST.COM
I20260812 03:03:33.480473 11681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:33.472485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58352 (local address 127.5.96.129:38237)
0812 03:03:33.472802 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:33.472811 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:33.472833 (+ 22us) server_negotiation.cc:407] Connection header received
0812 03:03:33.472878 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:33.472882 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:33.473036 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:33.473144 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:33.473711 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:33.474441 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:33.475190 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:33.475526 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:33.475602 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:33.477678 (+ 2076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:33.477687 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:33.477697 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:33.477729 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:33.478993 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:33.479343 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:33.479347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:33.479348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:33.479419 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:33.479598 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:33.479600 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:33.479601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:33.479867 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:33.480020 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:33.480174 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 03:03:33.480301 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":116,"threads_started":1}
Aug 12 03:03:33 dist-test-slave-6bbx krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:33.489873 11027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:33.482895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51458 (local address 127.5.96.190:45859)
0812 03:03:33.483007 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:33.483011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:33.483055 (+ 44us) server_negotiation.cc:407] Connection header received
0812 03:03:33.483181 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:33.483183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:33.483220 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:33.483283 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:33.483875 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:33.484354 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:33.485082 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:33.485290 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:33.485376 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:33.487303 (+ 1927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:33.487307 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:33.487309 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:33.487332 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:33.488394 (+ 1062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:33.488761 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:33.488764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:33.488765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:33.488836 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:33.489074 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:33.489077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:33.489079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:33.489279 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:33.489396 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:33.489633 (+ 237us) server_negotiation.cc:299] Negotiation successful
0812 03:03:33.489747 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260812 03:03:33.490200 10968 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51458
I20260812 03:03:33.490705 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 11064
I20260812 03:03:33.532290 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 11273
I20260812 03:03:33.547065 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 11477
I20260812 03:03:33.561576 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 10924
2026-08-12T03:03:33Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2254 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:03:33 dist-test-slave-6bbx krb5kdc[11687](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:03:33 dist-test-slave-6bbx krb5kdc[11687](info): set up 2 sockets
Aug 12 03:03:33 dist-test-slave-6bbx krb5kdc[11687](info): commencing operation
krb5kdc: starting...
W20260812 03:03:34.716125 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:03:34 dist-test-slave-6bbx krb5kdc[11687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:03:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:03:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:03:34.828271 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:40967
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:39347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:40967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:34.937287 11713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:34.937507 11713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:34.937544 11713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:34.941344 11713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:03:34.941383 11713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:34.941401 11713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:34.941419 11713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:03:34.941432 11713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:03:34.946298 11713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:39347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:40967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:40967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:34.947118 11713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:34.948017 11713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:34.955510 11719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:34.955695 11713 server_base.cc:1034] running on GCE node
W20260812 03:03:34.955754 11724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:34.955525 11720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:34.956348 11713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:34.957332 11713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:34.958488 11713 hybrid_clock.cc:648] HybridClock initialized: now 1786503814958464 us; error 35 us; skew 500 ppm
Aug 12 03:03:34 dist-test-slave-6bbx krb5kdc[11687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503814, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:34.964898 11713 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:03:34.965603 11713 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:34.965673 11713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:34.965883 11713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:34.967806 11713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "b2115382d34b4c6ea9a5b574a828f529"
format_stamp: "Formatted at 2026-08-12 03:03:34 on dist-test-slave-6bbx"
server_key: "720f465feeb56dbfbaa996373126ed9e"
server_key_iv: "c59737f6afce52a0602edbb6db407ebf"
server_key_version: "encryptionkey@0"
I20260812 03:03:34.968328 11713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "b2115382d34b4c6ea9a5b574a828f529"
format_stamp: "Formatted at 2026-08-12 03:03:34 on dist-test-slave-6bbx"
server_key: "720f465feeb56dbfbaa996373126ed9e"
server_key_iv: "c59737f6afce52a0602edbb6db407ebf"
server_key_version: "encryptionkey@0"
I20260812 03:03:34.971630 11713 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 03:03:34.973845 11732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:34.974933 11713 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:34.975042 11713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "b2115382d34b4c6ea9a5b574a828f529"
format_stamp: "Formatted at 2026-08-12 03:03:34 on dist-test-slave-6bbx"
server_key: "720f465feeb56dbfbaa996373126ed9e"
server_key_iv: "c59737f6afce52a0602edbb6db407ebf"
server_key_version: "encryptionkey@0"
I20260812 03:03:34.975126 11713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:34.984239 11713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:34.986615 11713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:34.986779 11713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:34.993297 11713 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:40967
I20260812 03:03:34.993532 11812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:40967 every 8 connection(s)
I20260812 03:03:34.993870 11713 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:03:34.996465 11815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:35.001636 11815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2115382d34b4c6ea9a5b574a828f529: Bootstrap starting.
I20260812 03:03:35.003440 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 11713
I20260812 03:03:35.003535 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:03:35.003818 5506 external_mini_cluster.cc:1428] Setting key 58256c75c49f47959083bc1d1b0cc7b4
I20260812 03:03:35.004042 11815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2115382d34b4c6ea9a5b574a828f529: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:35.004961 11815 log.cc:824] T 00000000000000000000000000000000 P b2115382d34b4c6ea9a5b574a828f529: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:35.006865 11815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2115382d34b4c6ea9a5b574a828f529: No bootstrap required, opened a new log
I20260812 03:03:35.010951 11815 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2115382d34b4c6ea9a5b574a828f529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2115382d34b4c6ea9a5b574a828f529" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40967 } }
I20260812 03:03:35.011217 11815 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2115382d34b4c6ea9a5b574a828f529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:35.011281 11815 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2115382d34b4c6ea9a5b574a828f529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2115382d34b4c6ea9a5b574a828f529, State: Initialized, Role: FOLLOWER
I20260812 03:03:35.011780 11815 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2115382d34b4c6ea9a5b574a828f529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2115382d34b4c6ea9a5b574a828f529" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40967 } }
I20260812 03:03:35.011926 11815 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2115382d34b4c6ea9a5b574a828f529 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:03:35.011988 11815 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2115382d34b4c6ea9a5b574a828f529 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:03:35.012081 11815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2115382d34b4c6ea9a5b574a828f529 [term 0 FOLLOWER]: Advancing to term 1
Aug 12 03:03:35 dist-test-slave-6bbx krb5kdc[11687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:35.012935 11815 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2115382d34b4c6ea9a5b574a828f529 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2115382d34b4c6ea9a5b574a828f529" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40967 } }
I20260812 03:03:35.013262 11815 leader_election.cc:302] T 00000000000000000000000000000000 P b2115382d34b4c6ea9a5b574a828f529 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2115382d34b4c6ea9a5b574a828f529; no voters:
I20260812 03:03:35.013572 11815 leader_election.cc:288] T 00000000000000000000000000000000 P b2115382d34b4c6ea9a5b574a828f529 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:03:35.013656 11820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2115382d34b4c6ea9a5b574a828f529 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:35.014117 11820 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2115382d34b4c6ea9a5b574a828f529 [term 1 LEADER]: Becoming Leader. State: Replica: b2115382d34b4c6ea9a5b574a828f529, State: Running, Role: LEADER
I20260812 03:03:35.014457 11820 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2115382d34b4c6ea9a5b574a828f529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2115382d34b4c6ea9a5b574a828f529" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40967 } }
I20260812 03:03:35.014792 11815 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2115382d34b4c6ea9a5b574a828f529 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:03:35.016060 11821 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2115382d34b4c6ea9a5b574a828f529 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2115382d34b4c6ea9a5b574a828f529" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2115382d34b4c6ea9a5b574a828f529" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40967 } } }
I20260812 03:03:35.016183 11821 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2115382d34b4c6ea9a5b574a828f529 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:35.016502 11822 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2115382d34b4c6ea9a5b574a828f529 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2115382d34b4c6ea9a5b574a828f529. Latest consensus state: current_term: 1 leader_uuid: "b2115382d34b4c6ea9a5b574a828f529" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2115382d34b4c6ea9a5b574a828f529" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40967 } } }
I20260812 03:03:35.016592 11822 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2115382d34b4c6ea9a5b574a828f529 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:35.017215 11844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:03:35.018740 11818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:35.004799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56260 (local address 127.5.96.190:40967)
0812 03:03:35.005170 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:35.005185 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:35.005219 (+ 34us) server_negotiation.cc:407] Connection header received
0812 03:03:35.005909 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:35.005928 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:35.006224 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:35.006490 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:35.007354 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.008336 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.009525 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.010131 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.010249 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:35.012889 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:35.012894 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:35.012905 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:35.012931 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:35.014730 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:35.015177 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:35.015183 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:35.015189 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:35.015304 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:35.016722 (+ 1418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:35.016726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:35.016728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:35.017231 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:35.017602 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:35.017730 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 03:03:35.018001 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":109,"threads_started":1}
I20260812 03:03:35.020200 11844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:03:35.028982 11844 catalog_manager.cc:1347] Generated new cluster ID: 0907c606a2e344a48df7b660856b39fd
I20260812 03:03:35.029033 11844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:03:35.042372 11844 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:03:35.043190 11844 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:03:35.048399 11844 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2115382d34b4c6ea9a5b574a828f529: Generated new TSK 0
I20260812 03:03:35.048959 11844 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.96.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
I20260812 03:03:35.082338 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:0
--local_ip_for_outbound_sockets=127.5.96.129
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:40967
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:39347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:35.189754 11858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:35.189977 11858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:35.190018 11858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:35.193931 11858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:35.193981 11858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:35.194046 11858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:03:35.199014 11858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:39347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:40967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:35.199940 11858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:35.200862 11858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:35.209509 11867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:35.209513 11872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:35.209542 11866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:35.209746 11858 server_base.cc:1034] running on GCE node
I20260812 03:03:35.210126 11858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:35.210739 11858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:35.211948 11858 hybrid_clock.cc:648] HybridClock initialized: now 1786503815211921 us; error 41 us; skew 500 ppm
Aug 12 03:03:35 dist-test-slave-6bbx krb5kdc[11687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503815, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:35.214558 11858 init.cc:377] Logged in from keytab as kudu/127.5.96.129@KRBTEST.COM (short username kudu)
I20260812 03:03:35.215268 11858 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:35.215334 11858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:35.215525 11858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:35.217597 11858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-0/data/instance:
uuid: "ce2ca3f201ea47f2a9338ce54df5ff87"
format_stamp: "Formatted at 2026-08-12 03:03:35 on dist-test-slave-6bbx"
server_key: "0262d0e175ca6eb1a6f5da61bab1e057"
server_key_iv: "f682f74126b23a478212e31bf1468f68"
server_key_version: "encryptionkey@0"
I20260812 03:03:35.218171 11858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance:
uuid: "ce2ca3f201ea47f2a9338ce54df5ff87"
format_stamp: "Formatted at 2026-08-12 03:03:35 on dist-test-slave-6bbx"
server_key: "0262d0e175ca6eb1a6f5da61bab1e057"
server_key_iv: "f682f74126b23a478212e31bf1468f68"
server_key_version: "encryptionkey@0"
I20260812 03:03:35.221623 11858 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260812 03:03:35.223821 11880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:35.224905 11858 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:35.225010 11858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "ce2ca3f201ea47f2a9338ce54df5ff87"
format_stamp: "Formatted at 2026-08-12 03:03:35 on dist-test-slave-6bbx"
server_key: "0262d0e175ca6eb1a6f5da61bab1e057"
server_key_iv: "f682f74126b23a478212e31bf1468f68"
server_key_version: "encryptionkey@0"
I20260812 03:03:35.225095 11858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:35.243301 11858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:35.246104 11858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:35.246268 11858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:35.246773 11858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:35.247531 11858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:35.247579 11858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:35.247629 11858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:35.247655 11858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:35.255925 11858 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:33271
I20260812 03:03:35.255951 12058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:33271 every 8 connection(s)
I20260812 03:03:35.256445 11858 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:03:35.258213 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 11858
I20260812 03:03:35.258322 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance
I20260812 03:03:35.258570 5506 external_mini_cluster.cc:1428] Setting key 2848facb5fe0449b8cdff04b909bca7d
Aug 12 03:03:35 dist-test-slave-6bbx krb5kdc[11687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503815, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.96.130@KRBTEST.COM; defaulting to no policy
I20260812 03:03:35.269838 11818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:35.257954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:54995 (local address 127.5.96.190:40967)
0812 03:03:35.258087 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:35.258092 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:35.258982 (+ 890us) server_negotiation.cc:407] Connection header received
0812 03:03:35.260264 (+ 1282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:35.260267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:35.260322 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:35.260402 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:35.261761 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.262388 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.263907 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.264222 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.264328 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:35.266598 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:35.266602 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:35.266603 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:35.266627 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:35.267889 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:35.268365 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:35.268368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:35.268369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:35.268447 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:35.268757 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:35.268760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:35.268762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:35.268958 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:35.269046 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:35.269513 (+ 467us) server_negotiation.cc:299] Negotiation successful
0812 03:03:35.269673 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:03:35.270536 12063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:35.258359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:40967 (local address 127.5.96.129:54995)
0812 03:03:35.258830 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:35.258881 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:35.260053 (+ 1172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:35.260545 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:35.260551 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:35.261020 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:35.261612 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:35.261629 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.262518 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.262521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:35.263774 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:35.263785 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.264346 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.264349 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:35.264479 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:35.264956 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:35.264971 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:35.266443 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:35.267994 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:35.268000 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:35.268014 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:35.268269 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:35.268532 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:35.268535 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:35.268537 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:35.268668 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:35.269057 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:35.269063 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:35.269295 (+ 232us) client_negotiation.cc:769] Sending connection context
0812 03:03:35.269453 (+ 158us) client_negotiation.cc:240] Negotiation successful
0812 03:03:35.269743 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":118,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.130@KRBTEST.COM" created.
I20260812 03:03:35.271720 12060 heartbeater.cc:347] Connected to a master server at 127.5.96.190:40967
I20260812 03:03:35.271967 12060 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:35.272457 12060 heartbeater.cc:511] Master 127.5.96.190:40967 requested a full tablet report, sending...
I20260812 03:03:35.274078 11756 ts_manager.cc:194] Registered new tserver with Master: ce2ca3f201ea47f2a9338ce54df5ff87 (127.5.96.129:33271)
I20260812 03:03:35.275597 11756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.129@KRBTEST.COM'} at 127.5.96.129:54995
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
I20260812 03:03:35.284106 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:0
--local_ip_for_outbound_sockets=127.5.96.130
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:40967
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:39347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:35.390290 12070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:35.390518 12070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:35.390563 12070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:35.394403 12070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:35.394449 12070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:35.394517 12070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:03:35.399396 12070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:39347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:40967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:35.400331 12070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:35.401238 12070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:35.409829 12081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:35.409881 12079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:35.409854 12078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:35.410106 12070 server_base.cc:1034] running on GCE node
I20260812 03:03:35.410490 12070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:35.411049 12070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:35.412199 12070 hybrid_clock.cc:648] HybridClock initialized: now 1786503815412185 us; error 28 us; skew 500 ppm
Aug 12 03:03:35 dist-test-slave-6bbx krb5kdc[11687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503815, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:35.414819 12070 init.cc:377] Logged in from keytab as kudu/127.5.96.130@KRBTEST.COM (short username kudu)
I20260812 03:03:35.415571 12070 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:35.415644 12070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:35.415880 12070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:35.417758 12070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-1/data/instance:
uuid: "f190bba4ca0d43c9a27c2da017a4bdc9"
format_stamp: "Formatted at 2026-08-12 03:03:35 on dist-test-slave-6bbx"
server_key: "89fe15afcc9fd3759d8d07a360507197"
server_key_iv: "72a3634faf2d1a1e8c9437a575fde2ae"
server_key_version: "encryptionkey@0"
I20260812 03:03:35.418315 12070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance:
uuid: "f190bba4ca0d43c9a27c2da017a4bdc9"
format_stamp: "Formatted at 2026-08-12 03:03:35 on dist-test-slave-6bbx"
server_key: "89fe15afcc9fd3759d8d07a360507197"
server_key_iv: "72a3634faf2d1a1e8c9437a575fde2ae"
server_key_version: "encryptionkey@0"
I20260812 03:03:35.421666 12070 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 03:03:35.423879 12091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:35.424969 12070 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:03:35.425065 12070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "f190bba4ca0d43c9a27c2da017a4bdc9"
format_stamp: "Formatted at 2026-08-12 03:03:35 on dist-test-slave-6bbx"
server_key: "89fe15afcc9fd3759d8d07a360507197"
server_key_iv: "72a3634faf2d1a1e8c9437a575fde2ae"
server_key_version: "encryptionkey@0"
I20260812 03:03:35.425155 12070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:35.435334 12070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:35.437848 12070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:35.438011 12070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:35.438524 12070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:35.439289 12070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:35.439337 12070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:35.439376 12070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:35.439405 12070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:35.447357 12070 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:37133
I20260812 03:03:35.447376 12273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:37133 every 8 connection(s)
I20260812 03:03:35.447890 12070 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:03:35.449200 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 12070
I20260812 03:03:35.449308 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance
I20260812 03:03:35.449625 5506 external_mini_cluster.cc:1428] Setting key a3d43f85e6b5f95fb7a72d894a7a5bbd
Aug 12 03:03:35 dist-test-slave-6bbx krb5kdc[11687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503815, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.96.131@KRBTEST.COM; defaulting to no policy
I20260812 03:03:35.461186 11818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:35.449502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:56603 (local address 127.5.96.190:40967)
0812 03:03:35.449659 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:35.449664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:35.450474 (+ 810us) server_negotiation.cc:407] Connection header received
0812 03:03:35.451908 (+ 1434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:35.451912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:35.451965 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:35.452047 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:35.453121 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.453669 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.455149 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.455365 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.455434 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:35.457973 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:35.457976 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:35.457978 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:35.458001 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:35.459239 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:35.459716 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:35.459720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:35.459722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:35.459798 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:35.460115 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:35.460117 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:35.460118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:35.460343 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:35.460436 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:35.460909 (+ 473us) server_negotiation.cc:299] Negotiation successful
0812 03:03:35.461031 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:03:35.461863 12276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:35.449854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:40967 (local address 127.5.96.130:56603)
0812 03:03:35.450317 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:35.450371 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:35.451688 (+ 1317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:35.452168 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:35.452174 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:35.452499 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:35.452954 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:35.452972 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.453791 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.453794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:35.455041 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:35.455053 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.455500 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.455504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:35.455718 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:35.456242 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:35.456261 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:35.457820 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:35.459347 (+ 1527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:35.459352 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:35.459366 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:35.459607 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:35.459881 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:35.459884 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:35.459886 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:35.460016 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:35.460447 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:35.460453 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:35.460668 (+ 215us) client_negotiation.cc:769] Sending connection context
0812 03:03:35.460826 (+ 158us) client_negotiation.cc:240] Negotiation successful
0812 03:03:35.461115 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":119,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.131@KRBTEST.COM" created.
I20260812 03:03:35.462914 12274 heartbeater.cc:347] Connected to a master server at 127.5.96.190:40967
I20260812 03:03:35.463155 12274 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:35.463626 12274 heartbeater.cc:511] Master 127.5.96.190:40967 requested a full tablet report, sending...
I20260812 03:03:35.464687 11756 ts_manager.cc:194] Registered new tserver with Master: f190bba4ca0d43c9a27c2da017a4bdc9 (127.5.96.130:37133)
I20260812 03:03:35.465209 11756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.130@KRBTEST.COM'} at 127.5.96.130:56603
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
I20260812 03:03:35.475306 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:0
--local_ip_for_outbound_sockets=127.5.96.131
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:40967
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:39347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:35.581707 12281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:35.581950 12281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:35.582010 12281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:35.585944 12281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:35.585999 12281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:35.586077 12281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:03:35.591049 12281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:39347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:40967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:35.591982 12281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:35.592937 12281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:35.601773 12292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:35.601791 12290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:35.601789 12296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:35.602041 12281 server_base.cc:1034] running on GCE node
I20260812 03:03:35.602449 12281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:35.603018 12281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:35.604168 12281 hybrid_clock.cc:648] HybridClock initialized: now 1786503815604147 us; error 31 us; skew 500 ppm
Aug 12 03:03:35 dist-test-slave-6bbx krb5kdc[11687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503815, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:35.606873 12281 init.cc:377] Logged in from keytab as kudu/127.5.96.131@KRBTEST.COM (short username kudu)
I20260812 03:03:35.607549 12281 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:35.607612 12281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:35.607825 12281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:35.609679 12281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-2/data/instance:
uuid: "0c04fe408fa5413ab93ab45aec8fbccf"
format_stamp: "Formatted at 2026-08-12 03:03:35 on dist-test-slave-6bbx"
server_key: "2efb57d47d2511477d892a7f503f4fc8"
server_key_iv: "ba20b54f2f6ddbee57151875fdbc8f5f"
server_key_version: "encryptionkey@0"
I20260812 03:03:35.610241 12281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance:
uuid: "0c04fe408fa5413ab93ab45aec8fbccf"
format_stamp: "Formatted at 2026-08-12 03:03:35 on dist-test-slave-6bbx"
server_key: "2efb57d47d2511477d892a7f503f4fc8"
server_key_iv: "ba20b54f2f6ddbee57151875fdbc8f5f"
server_key_version: "encryptionkey@0"
I20260812 03:03:35.613862 12281 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 03:03:35.616235 12306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:35.617386 12281 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:03:35.617499 12281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "0c04fe408fa5413ab93ab45aec8fbccf"
format_stamp: "Formatted at 2026-08-12 03:03:35 on dist-test-slave-6bbx"
server_key: "2efb57d47d2511477d892a7f503f4fc8"
server_key_iv: "ba20b54f2f6ddbee57151875fdbc8f5f"
server_key_version: "encryptionkey@0"
I20260812 03:03:35.617586 12281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:35.633661 12281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:35.636082 12281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:35.636243 12281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:35.636741 12281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:35.637485 12281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:35.637532 12281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:35.637580 12281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:35.637606 12281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:35.645732 12281 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:33641
I20260812 03:03:35.645762 12491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:33641 every 8 connection(s)
I20260812 03:03:35.646240 12281 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:03:35.650370 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 12281
I20260812 03:03:35.650461 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance
I20260812 03:03:35.650724 5506 external_mini_cluster.cc:1428] Setting key 04d17dfe570f3b6d57a300557a1565e2
Aug 12 03:03:35 dist-test-slave-6bbx krb5kdc[11687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503815, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:35.658483 11818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:35.647800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:40519 (local address 127.5.96.190:40967)
0812 03:03:35.647923 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:35.647926 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:35.649006 (+ 1080us) server_negotiation.cc:407] Connection header received
0812 03:03:35.649825 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:35.649829 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:35.649888 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:35.649959 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:35.651088 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.651793 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.652878 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.653103 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.653189 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:35.655344 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:35.655348 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:35.655351 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:35.655378 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:35.656640 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:35.657103 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:35.657107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:35.657108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:35.657185 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:35.657487 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:35.657491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:35.657492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:35.657662 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:35.657747 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:35.658200 (+ 453us) server_negotiation.cc:299] Negotiation successful
0812 03:03:35.658327 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:03:35.659157 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:35.648094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:40967 (local address 127.5.96.131:40519)
0812 03:03:35.648866 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:35.648903 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:35.649619 (+ 716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:35.650130 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:35.650136 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:35.650460 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:35.650941 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:35.650956 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.651916 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.651919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:35.652759 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:35.652768 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.653233 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.653237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:35.653390 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:35.653876 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:35.653891 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:35.655189 (+ 1298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:35.656755 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:35.656761 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:35.656774 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:35.657009 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:35.657265 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:35.657268 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:35.657270 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:35.657401 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:35.657766 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:35.657773 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:35.657983 (+ 210us) client_negotiation.cc:769] Sending connection context
0812 03:03:35.658139 (+ 156us) client_negotiation.cc:240] Negotiation successful
0812 03:03:35.658425 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":118,"threads_started":1}
I20260812 03:03:35.660105 12492 heartbeater.cc:347] Connected to a master server at 127.5.96.190:40967
I20260812 03:03:35.660303 12492 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:35.660687 12492 heartbeater.cc:511] Master 127.5.96.190:40967 requested a full tablet report, sending...
I20260812 03:03:35.661581 11756 ts_manager.cc:194] Registered new tserver with Master: 0c04fe408fa5413ab93ab45aec8fbccf (127.5.96.131:33641)
I20260812 03:03:35.662127 11756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.131@KRBTEST.COM'} at 127.5.96.131:40519
I20260812 03:03:35.664093 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:03:35.671794 11818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:35.665776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56274 (local address 127.5.96.190:40967)
0812 03:03:35.665904 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:35.665908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:35.666059 (+ 151us) server_negotiation.cc:407] Connection header received
0812 03:03:35.666149 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:35.666151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:35.666187 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:35.666255 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:35.666814 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.667275 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.668052 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.668252 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.668321 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:35.669389 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:35.669392 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:35.669393 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:35.669414 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:35.670587 (+ 1173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:35.670895 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:35.670897 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:35.670898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:35.670961 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:35.671131 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:35.671132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:35.671133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:35.671323 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:35.671417 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:35.671560 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 03:03:35.671637 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 03:03:35.674100 11756 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56274:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:03:35.676992 11756 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:03:35.690210 12510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:35.684791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:33271 (local address 127.0.0.1:40420)
0812 03:03:35.685411 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:35.685425 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:35.685517 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:35.686308 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:35.686314 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:35.686338 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:35.686453 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:35.686460 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.687580 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.687582 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:35.688812 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:35.688823 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.689701 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.689705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:35.689833 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:35.689873 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 03:03:35.689965 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 03:03:35.690079 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":91,"threads_started":1}
I20260812 03:03:35.690234 12511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:35.685437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40420 (local address 127.5.96.129:33271)
0812 03:03:35.685833 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:35.685844 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:35.685867 (+ 23us) server_negotiation.cc:407] Connection header received
0812 03:03:35.685944 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:35.685949 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:35.686144 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:35.686610 (+ 466us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:35.686777 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.687456 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.688950 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.689592 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.689688 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:35.689833 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:35.690012 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 03:03:35.690100 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":123,"threads_started":1}
I20260812 03:03:35.690444 12508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:35.685179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:37133 (local address 127.0.0.1:34112)
0812 03:03:35.685505 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:35.685516 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:35.685596 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:35.686517 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:35.686520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:35.686538 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:35.686624 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:35.686629 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.688067 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.688070 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:35.688992 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:35.689002 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.690074 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.690078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:35.690176 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:35.690197 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 03:03:35.690246 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 03:03:35.690313 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":11,"thread_start_us":89,"threads_started":1}
I20260812 03:03:35.690850 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:35.684669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:33641 (local address 127.0.0.1:36408)
0812 03:03:35.685123 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:35.685165 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:35.685286 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:35.686413 (+ 1127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:35.686416 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:35.686433 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:35.686528 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:35.686534 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.688033 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.688036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:35.689117 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:35.689126 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.690074 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.690078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:35.690555 (+ 477us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:35.690578 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:03:35.690652 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 03:03:35.690731 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":392,"thread_start_us":77,"threads_started":1}
I20260812 03:03:35.690976 12509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:35.684781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36408 (local address 127.5.96.131:33641)
0812 03:03:35.685997 (+ 1216us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:35.686007 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:35.686024 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:03:35.686082 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:35.686086 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:35.686255 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:35.686658 (+ 403us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:35.686796 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.687896 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.689213 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.689934 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.690124 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:35.690270 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:35.690772 (+ 502us) server_negotiation.cc:299] Negotiation successful
0812 03:03:35.690846 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1133,"thread_start_us":103,"threads_started":1}
I20260812 03:03:35.691524 12512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:35.685587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34112 (local address 127.5.96.130:37133)
0812 03:03:35.685871 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:35.685882 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:35.685905 (+ 23us) server_negotiation.cc:407] Connection header received
0812 03:03:35.685978 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:35.685983 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:35.686401 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:35.686545 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:35.686699 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.687896 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.689127 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.689934 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.691150 (+ 1216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:35.691265 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:35.691305 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 03:03:35.691380 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":94,"threads_started":1}
I20260812 03:03:35.693318 12390 tablet_service.cc:1467] Processing CreateTablet for tablet 8e96938a81ff4cae919ec735b9b4a39e (DEFAULT_TABLE table=test-table [id=9def9d6ac7cf4b8a928e9eafab9e67c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:35.693325 11954 tablet_service.cc:1467] Processing CreateTablet for tablet 8e96938a81ff4cae919ec735b9b4a39e (DEFAULT_TABLE table=test-table [id=9def9d6ac7cf4b8a928e9eafab9e67c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:35.694195 11954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e96938a81ff4cae919ec735b9b4a39e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:35.694201 12390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e96938a81ff4cae919ec735b9b4a39e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:35.698370 12164 tablet_service.cc:1467] Processing CreateTablet for tablet 8e96938a81ff4cae919ec735b9b4a39e (DEFAULT_TABLE table=test-table [id=9def9d6ac7cf4b8a928e9eafab9e67c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:35.699373 12164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e96938a81ff4cae919ec735b9b4a39e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:35.699990 12513 tablet_bootstrap.cc:492] T 8e96938a81ff4cae919ec735b9b4a39e P ce2ca3f201ea47f2a9338ce54df5ff87: Bootstrap starting.
I20260812 03:03:35.701236 12514 tablet_bootstrap.cc:492] T 8e96938a81ff4cae919ec735b9b4a39e P 0c04fe408fa5413ab93ab45aec8fbccf: Bootstrap starting.
I20260812 03:03:35.702724 12513 tablet_bootstrap.cc:654] T 8e96938a81ff4cae919ec735b9b4a39e P ce2ca3f201ea47f2a9338ce54df5ff87: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:35.703285 12514 tablet_bootstrap.cc:654] T 8e96938a81ff4cae919ec735b9b4a39e P 0c04fe408fa5413ab93ab45aec8fbccf: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:35.703598 12513 log.cc:824] T 8e96938a81ff4cae919ec735b9b4a39e P ce2ca3f201ea47f2a9338ce54df5ff87: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:35.704384 12514 log.cc:824] T 8e96938a81ff4cae919ec735b9b4a39e P 0c04fe408fa5413ab93ab45aec8fbccf: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:35.705575 12513 tablet_bootstrap.cc:492] T 8e96938a81ff4cae919ec735b9b4a39e P ce2ca3f201ea47f2a9338ce54df5ff87: No bootstrap required, opened a new log
I20260812 03:03:35.705770 12513 ts_tablet_manager.cc:1397] T 8e96938a81ff4cae919ec735b9b4a39e P ce2ca3f201ea47f2a9338ce54df5ff87: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:03:35.706033 12522 tablet_bootstrap.cc:492] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9: Bootstrap starting.
I20260812 03:03:35.706220 12514 tablet_bootstrap.cc:492] T 8e96938a81ff4cae919ec735b9b4a39e P 0c04fe408fa5413ab93ab45aec8fbccf: No bootstrap required, opened a new log
I20260812 03:03:35.706399 12514 ts_tablet_manager.cc:1397] T 8e96938a81ff4cae919ec735b9b4a39e P 0c04fe408fa5413ab93ab45aec8fbccf: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260812 03:03:35.708004 12522 tablet_bootstrap.cc:654] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:35.708909 12522 log.cc:824] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:35.709476 12513 raft_consensus.cc:348] T 8e96938a81ff4cae919ec735b9b4a39e P ce2ca3f201ea47f2a9338ce54df5ff87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c04fe408fa5413ab93ab45aec8fbccf" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 33641 } } peers { permanent_uuid: "ce2ca3f201ea47f2a9338ce54df5ff87" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33271 } } peers { permanent_uuid: "f190bba4ca0d43c9a27c2da017a4bdc9" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 37133 } }
I20260812 03:03:35.709743 12513 raft_consensus.cc:374] T 8e96938a81ff4cae919ec735b9b4a39e P ce2ca3f201ea47f2a9338ce54df5ff87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:35.709802 12513 raft_consensus.cc:729] T 8e96938a81ff4cae919ec735b9b4a39e P ce2ca3f201ea47f2a9338ce54df5ff87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce2ca3f201ea47f2a9338ce54df5ff87, State: Initialized, Role: FOLLOWER
I20260812 03:03:35.709966 12514 raft_consensus.cc:348] T 8e96938a81ff4cae919ec735b9b4a39e P 0c04fe408fa5413ab93ab45aec8fbccf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c04fe408fa5413ab93ab45aec8fbccf" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 33641 } } peers { permanent_uuid: "ce2ca3f201ea47f2a9338ce54df5ff87" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33271 } } peers { permanent_uuid: "f190bba4ca0d43c9a27c2da017a4bdc9" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 37133 } }
I20260812 03:03:35.710208 12514 raft_consensus.cc:374] T 8e96938a81ff4cae919ec735b9b4a39e P 0c04fe408fa5413ab93ab45aec8fbccf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:35.710261 12514 raft_consensus.cc:729] T 8e96938a81ff4cae919ec735b9b4a39e P 0c04fe408fa5413ab93ab45aec8fbccf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c04fe408fa5413ab93ab45aec8fbccf, State: Initialized, Role: FOLLOWER
I20260812 03:03:35.710229 12513 consensus_queue.cc:260] T 8e96938a81ff4cae919ec735b9b4a39e P ce2ca3f201ea47f2a9338ce54df5ff87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c04fe408fa5413ab93ab45aec8fbccf" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 33641 } } peers { permanent_uuid: "ce2ca3f201ea47f2a9338ce54df5ff87" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33271 } } peers { permanent_uuid: "f190bba4ca0d43c9a27c2da017a4bdc9" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 37133 } }
I20260812 03:03:35.710526 12522 tablet_bootstrap.cc:492] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9: No bootstrap required, opened a new log
I20260812 03:03:35.710711 12522 ts_tablet_manager.cc:1397] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 03:03:35.710671 12514 consensus_queue.cc:260] T 8e96938a81ff4cae919ec735b9b4a39e P 0c04fe408fa5413ab93ab45aec8fbccf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c04fe408fa5413ab93ab45aec8fbccf" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 33641 } } peers { permanent_uuid: "ce2ca3f201ea47f2a9338ce54df5ff87" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33271 } } peers { permanent_uuid: "f190bba4ca0d43c9a27c2da017a4bdc9" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 37133 } }
I20260812 03:03:35.710992 12060 heartbeater.cc:503] Master 127.5.96.190:40967 was elected leader, sending a full tablet report...
I20260812 03:03:35.711180 12513 ts_tablet_manager.cc:1428] T 8e96938a81ff4cae919ec735b9b4a39e P ce2ca3f201ea47f2a9338ce54df5ff87: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 03:03:35.711966 12514 ts_tablet_manager.cc:1428] T 8e96938a81ff4cae919ec735b9b4a39e P 0c04fe408fa5413ab93ab45aec8fbccf: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260812 03:03:35.712015 12492 heartbeater.cc:503] Master 127.5.96.190:40967 was elected leader, sending a full tablet report...
I20260812 03:03:35.715119 12522 raft_consensus.cc:348] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c04fe408fa5413ab93ab45aec8fbccf" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 33641 } } peers { permanent_uuid: "ce2ca3f201ea47f2a9338ce54df5ff87" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33271 } } peers { permanent_uuid: "f190bba4ca0d43c9a27c2da017a4bdc9" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 37133 } }
I20260812 03:03:35.715404 12522 raft_consensus.cc:374] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:35.715464 12522 raft_consensus.cc:729] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f190bba4ca0d43c9a27c2da017a4bdc9, State: Initialized, Role: FOLLOWER
I20260812 03:03:35.715867 12522 consensus_queue.cc:260] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c04fe408fa5413ab93ab45aec8fbccf" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 33641 } } peers { permanent_uuid: "ce2ca3f201ea47f2a9338ce54df5ff87" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33271 } } peers { permanent_uuid: "f190bba4ca0d43c9a27c2da017a4bdc9" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 37133 } }
I20260812 03:03:35.716435 12274 heartbeater.cc:503] Master 127.5.96.190:40967 was elected leader, sending a full tablet report...
I20260812 03:03:35.716647 12522 ts_tablet_manager.cc:1428] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 03:03:35.723037 12529 raft_consensus.cc:482] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:03:35.723170 12529 raft_consensus.cc:504] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c04fe408fa5413ab93ab45aec8fbccf" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 33641 } } peers { permanent_uuid: "ce2ca3f201ea47f2a9338ce54df5ff87" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33271 } } peers { permanent_uuid: "f190bba4ca0d43c9a27c2da017a4bdc9" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 37133 } }
I20260812 03:03:35.724108 12529 leader_election.cc:288] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c04fe408fa5413ab93ab45aec8fbccf (127.5.96.131:33641), ce2ca3f201ea47f2a9338ce54df5ff87 (127.5.96.129:33271)
I20260812 03:03:35.728667 12509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:35.724369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:53947 (local address 127.5.96.131:33641)
0812 03:03:35.724505 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:35.724510 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:35.724587 (+ 77us) server_negotiation.cc:407] Connection header received
0812 03:03:35.724690 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:35.724692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:35.724729 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:35.724820 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:35.725052 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.725811 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.727387 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.728017 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.728132 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:35.728191 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:35.728477 (+ 286us) server_negotiation.cc:299] Negotiation successful
0812 03:03:35.728561 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 03:03:35.728667 12511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:35.724424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:45863 (local address 127.5.96.129:33271)
0812 03:03:35.724586 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:35.724590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:35.724745 (+ 155us) server_negotiation.cc:407] Connection header received
0812 03:03:35.724832 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:35.724834 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:35.724875 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:35.724939 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:35.725145 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.725851 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.727386 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.727997 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.728113 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:35.728165 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:35.728477 (+ 312us) server_negotiation.cc:299] Negotiation successful
0812 03:03:35.728551 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260812 03:03:35.728677 12276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:35.724310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:33641 (local address 127.5.96.130:53947)
0812 03:03:35.724506 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:35.724520 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:35.724612 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:35.724834 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:35.724837 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:35.724851 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:35.724954 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:35.724960 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.725907 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.725910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:35.727269 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:35.727280 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.728125 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.728129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:35.728317 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:35.728362 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 03:03:35.728435 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 03:03:35.728522 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":141}
I20260812 03:03:35.728796 12530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:35.724340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:33271 (local address 127.5.96.130:45863)
0812 03:03:35.724665 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:35.724677 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:35.724759 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:35.724954 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:35.724957 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:35.724970 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:35.725062 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:35.725068 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.725926 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.725929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:35.727269 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:35.727281 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.728136 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.728140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:35.728312 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:35.728368 (+ 56us) client_negotiation.cc:769] Sending connection context
0812 03:03:35.728445 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 03:03:35.728522 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":106,"thread_start_us":90,"threads_started":1}
I20260812 03:03:35.729290 12419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e96938a81ff4cae919ec735b9b4a39e" candidate_uuid: "f190bba4ca0d43c9a27c2da017a4bdc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c04fe408fa5413ab93ab45aec8fbccf" is_pre_election: true
I20260812 03:03:35.729287 11989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e96938a81ff4cae919ec735b9b4a39e" candidate_uuid: "f190bba4ca0d43c9a27c2da017a4bdc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce2ca3f201ea47f2a9338ce54df5ff87" is_pre_election: true
I20260812 03:03:35.729545 11989 raft_consensus.cc:2454] T 8e96938a81ff4cae919ec735b9b4a39e P ce2ca3f201ea47f2a9338ce54df5ff87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f190bba4ca0d43c9a27c2da017a4bdc9 in term 0.
I20260812 03:03:35.729545 12419 raft_consensus.cc:2454] T 8e96938a81ff4cae919ec735b9b4a39e P 0c04fe408fa5413ab93ab45aec8fbccf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f190bba4ca0d43c9a27c2da017a4bdc9 in term 0.
I20260812 03:03:35.729977 12092 leader_election.cc:302] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0c04fe408fa5413ab93ab45aec8fbccf, ce2ca3f201ea47f2a9338ce54df5ff87, f190bba4ca0d43c9a27c2da017a4bdc9; no voters:
I20260812 03:03:35.730201 12529 raft_consensus.cc:2781] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:03:35.730281 12529 raft_consensus.cc:482] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:03:35.730332 12529 raft_consensus.cc:3037] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:35.731225 12529 raft_consensus.cc:504] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c04fe408fa5413ab93ab45aec8fbccf" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 33641 } } peers { permanent_uuid: "ce2ca3f201ea47f2a9338ce54df5ff87" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33271 } } peers { permanent_uuid: "f190bba4ca0d43c9a27c2da017a4bdc9" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 37133 } }
I20260812 03:03:35.731621 12529 leader_election.cc:288] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9 [CANDIDATE]: Term 1 election: Requested vote from peers 0c04fe408fa5413ab93ab45aec8fbccf (127.5.96.131:33641), ce2ca3f201ea47f2a9338ce54df5ff87 (127.5.96.129:33271)
I20260812 03:03:35.731824 12419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e96938a81ff4cae919ec735b9b4a39e" candidate_uuid: "f190bba4ca0d43c9a27c2da017a4bdc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c04fe408fa5413ab93ab45aec8fbccf"
I20260812 03:03:35.731834 11989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e96938a81ff4cae919ec735b9b4a39e" candidate_uuid: "f190bba4ca0d43c9a27c2da017a4bdc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce2ca3f201ea47f2a9338ce54df5ff87"
I20260812 03:03:35.731931 12419 raft_consensus.cc:3037] T 8e96938a81ff4cae919ec735b9b4a39e P 0c04fe408fa5413ab93ab45aec8fbccf [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:35.731935 11989 raft_consensus.cc:3037] T 8e96938a81ff4cae919ec735b9b4a39e P ce2ca3f201ea47f2a9338ce54df5ff87 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:35.732858 12419 raft_consensus.cc:2454] T 8e96938a81ff4cae919ec735b9b4a39e P 0c04fe408fa5413ab93ab45aec8fbccf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f190bba4ca0d43c9a27c2da017a4bdc9 in term 1.
I20260812 03:03:35.732897 11989 raft_consensus.cc:2454] T 8e96938a81ff4cae919ec735b9b4a39e P ce2ca3f201ea47f2a9338ce54df5ff87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f190bba4ca0d43c9a27c2da017a4bdc9 in term 1.
I20260812 03:03:35.733127 12097 leader_election.cc:302] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c04fe408fa5413ab93ab45aec8fbccf, f190bba4ca0d43c9a27c2da017a4bdc9; no voters:
I20260812 03:03:35.733290 12529 raft_consensus.cc:2781] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:35.733470 12529 raft_consensus.cc:686] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9 [term 1 LEADER]: Becoming Leader. State: Replica: f190bba4ca0d43c9a27c2da017a4bdc9, State: Running, Role: LEADER
I20260812 03:03:35.733719 12529 consensus_queue.cc:237] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c04fe408fa5413ab93ab45aec8fbccf" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 33641 } } peers { permanent_uuid: "ce2ca3f201ea47f2a9338ce54df5ff87" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33271 } } peers { permanent_uuid: "f190bba4ca0d43c9a27c2da017a4bdc9" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 37133 } }
I20260812 03:03:35.736263 11756 catalog_manager.cc:5495] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9 reported cstate change: term changed from 0 to 1, leader changed from <none> to f190bba4ca0d43c9a27c2da017a4bdc9 (127.5.96.130). New cstate: current_term: 1 leader_uuid: "f190bba4ca0d43c9a27c2da017a4bdc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c04fe408fa5413ab93ab45aec8fbccf" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 33641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce2ca3f201ea47f2a9338ce54df5ff87" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f190bba4ca0d43c9a27c2da017a4bdc9" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 37133 } health_report { overall_health: HEALTHY } } }
I20260812 03:03:35.751360 12512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:35.748162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34120 (local address 127.5.96.130:37133)
0812 03:03:35.748279 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:35.748283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:35.748296 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:03:35.748427 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:35.748429 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:35.748476 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:35.748555 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:03:35.748800 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.749264 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.750078 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:35.750288 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:35.750372 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:35.750644 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:03:35.750987 (+ 343us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:03:35.751086 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:35.751169 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 03:03:35.751228 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 03:03:35.756158 12419 raft_consensus.cc:1261] T 8e96938a81ff4cae919ec735b9b4a39e P 0c04fe408fa5413ab93ab45aec8fbccf [term 1 FOLLOWER]: Refusing update from remote peer f190bba4ca0d43c9a27c2da017a4bdc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:35.756160 11989 raft_consensus.cc:1261] T 8e96938a81ff4cae919ec735b9b4a39e P ce2ca3f201ea47f2a9338ce54df5ff87 [term 1 FOLLOWER]: Refusing update from remote peer f190bba4ca0d43c9a27c2da017a4bdc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:35.756768 12536 consensus_queue.cc:1035] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce2ca3f201ea47f2a9338ce54df5ff87" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:35.756922 12529 consensus_queue.cc:1035] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c04fe408fa5413ab93ab45aec8fbccf" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 33641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 03:03:35.757356 12062 tablet.cc:2369] T 8e96938a81ff4cae919ec735b9b4a39e P ce2ca3f201ea47f2a9338ce54df5ff87: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:03:35.762317 12539 mvcc.cc:205] Tried to move back new op lower bound from 7317519629331705856 to 7317519629247619072. Current Snapshot: MvccSnapshot[applied={T|T < 7317519629331705856}]
I20260812 03:03:35.763839 12540 mvcc.cc:205] Tried to move back new op lower bound from 7317519629331705856 to 7317519629247619072. Current Snapshot: MvccSnapshot[applied={T|T < 7317519629331705856}]
I20260812 03:03:35.765786 12541 mvcc.cc:205] Tried to move back new op lower bound from 7317519629331705856 to 7317519629247619072. Current Snapshot: MvccSnapshot[applied={T|T < 7317519629331705856 or (T in {7317519629331705856})}]
I20260812 03:03:35.771528 11754 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56274:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:35.771657 11754 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56274:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:35.773731 11754 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2115382d34b4c6ea9a5b574a828f529: Sending DeleteTablet for 3 replicas of tablet 8e96938a81ff4cae919ec735b9b4a39e
I20260812 03:03:35.774389 12390 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e96938a81ff4cae919ec735b9b4a39e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:35 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:36408
I20260812 03:03:35.774387 11954 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e96938a81ff4cae919ec735b9b4a39e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:35 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:40420
I20260812 03:03:35.774812 12553 tablet_replica.cc:330] T 8e96938a81ff4cae919ec735b9b4a39e P ce2ca3f201ea47f2a9338ce54df5ff87: stopping tablet replica
I20260812 03:03:35.774816 12164 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e96938a81ff4cae919ec735b9b4a39e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:35 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:34112
I20260812 03:03:35.775105 12553 raft_consensus.cc:2229] T 8e96938a81ff4cae919ec735b9b4a39e P ce2ca3f201ea47f2a9338ce54df5ff87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:35.775240 12557 tablet_replica.cc:330] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9: stopping tablet replica
I20260812 03:03:35.775399 12553 raft_consensus.cc:2258] T 8e96938a81ff4cae919ec735b9b4a39e P ce2ca3f201ea47f2a9338ce54df5ff87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:35.775610 12557 raft_consensus.cc:2229] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:03:35.775945 12554 tablet_replica.cc:330] T 8e96938a81ff4cae919ec735b9b4a39e P 0c04fe408fa5413ab93ab45aec8fbccf: stopping tablet replica
I20260812 03:03:35.776021 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 11858
I20260812 03:03:35.776862 12554 raft_consensus.cc:2229] T 8e96938a81ff4cae919ec735b9b4a39e P 0c04fe408fa5413ab93ab45aec8fbccf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:35.776058 12557 raft_consensus.cc:2258] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:35.777132 12554 raft_consensus.cc:2258] T 8e96938a81ff4cae919ec735b9b4a39e P 0c04fe408fa5413ab93ab45aec8fbccf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:35.781715 12557 ts_tablet_manager.cc:1905] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:35.784178 12557 ts_tablet_manager.cc:1918] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:03:35.784283 12557 log.cc:1196] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-1/wal/wals/8e96938a81ff4cae919ec735b9b4a39e
I20260812 03:03:35.784617 12557 ts_tablet_manager.cc:1939] T 8e96938a81ff4cae919ec735b9b4a39e P f190bba4ca0d43c9a27c2da017a4bdc9: Deleting consensus metadata
I20260812 03:03:35.785975 12554 ts_tablet_manager.cc:1905] T 8e96938a81ff4cae919ec735b9b4a39e P 0c04fe408fa5413ab93ab45aec8fbccf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:35.787925 11736 catalog_manager.cc:4841] TS f190bba4ca0d43c9a27c2da017a4bdc9 (127.5.96.130:37133): tablet 8e96938a81ff4cae919ec735b9b4a39e (table test-table [id=9def9d6ac7cf4b8a928e9eafab9e67c1]) successfully deleted
I20260812 03:03:35.788456 12554 ts_tablet_manager.cc:1918] T 8e96938a81ff4cae919ec735b9b4a39e P 0c04fe408fa5413ab93ab45aec8fbccf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:03:35.788555 12554 log.cc:1196] T 8e96938a81ff4cae919ec735b9b4a39e P 0c04fe408fa5413ab93ab45aec8fbccf: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786503790637001-5506-0/minicluster-data/ts-2/wal/wals/8e96938a81ff4cae919ec735b9b4a39e
I20260812 03:03:35.788903 12554 ts_tablet_manager.cc:1939] T 8e96938a81ff4cae919ec735b9b4a39e P 0c04fe408fa5413ab93ab45aec8fbccf: Deleting consensus metadata
I20260812 03:03:35.789978 11735 catalog_manager.cc:4841] TS 0c04fe408fa5413ab93ab45aec8fbccf (127.5.96.131:33641): tablet 8e96938a81ff4cae919ec735b9b4a39e (table test-table [id=9def9d6ac7cf4b8a928e9eafab9e67c1]) successfully deleted
W20260812 03:03:35.791245 11733 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.96.129:33271 (error 108)
I20260812 03:03:35.792055 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 12070
I20260812 03:03:35.792271 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:35.791970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:33271 (local address 127.0.0.1:40424)
0812 03:03:35.792106 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:35.792181 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.129:33271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 03:03:35.811883 11733 catalog_manager.cc:4584] TS ce2ca3f201ea47f2a9338ce54df5ff87 (127.5.96.129:33271): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8e96938a81ff4cae919ec735b9b4a39e: Network error: Client connection negotiation failed: client connection to 127.5.96.129:33271: connect: Connection refused (error 111)
I20260812 03:03:35.817291 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 12281
I20260812 03:03:35.827811 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 11713
2026-08-12T03:03:35Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2259 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T03:03:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:03:35Z Disabled control of system clock
I20260812 03:03:35.861249 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:38667
--webserver_interface=127.5.96.190
--webserver_port=0
--builtin_ntp_servers=127.5.96.148:40961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:38667
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 03:03:35.968989 12564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:35.969214 12564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:35.969249 12564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:35.973198 12564 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:03:35.973245 12564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:35.973259 12564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:35.973273 12564 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:03:35.973292 12564 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:03:35.977823 12564 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:38667
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:38667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:35.978685 12564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:35.979939 12564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:35.987710 12572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:35.987716 12573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:35.987828 12564 server_base.cc:1034] running on GCE node
W20260812 03:03:35.987720 12576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:35.988510 12564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:35.989483 12564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:35.990638 12564 hybrid_clock.cc:648] HybridClock initialized: now 1786503815990615 us; error 32 us; skew 500 ppm
I20260812 03:03:35.992321 12564 webserver.cc:460] Webserver started at http://127.5.96.190:33327/ using document root <none> and password file <none>
I20260812 03:03:35.992832 12564 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:35.992897 12564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:35.993089 12564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:35.994972 12564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "0672a00262e644de94a6f08524c678e6"
format_stamp: "Formatted at 2026-08-12 03:03:35 on dist-test-slave-6bbx"
server_key: "c552a11161c6bbe7cfd60b0daa7b7a87"
server_key_iv: "d9267fb89ffdde79c665afd25cf37d9c"
server_key_version: "encryptionkey@0"
I20260812 03:03:35.995523 12564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "0672a00262e644de94a6f08524c678e6"
format_stamp: "Formatted at 2026-08-12 03:03:35 on dist-test-slave-6bbx"
server_key: "c552a11161c6bbe7cfd60b0daa7b7a87"
server_key_iv: "d9267fb89ffdde79c665afd25cf37d9c"
server_key_version: "encryptionkey@0"
I20260812 03:03:35.998844 12564 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:03:36.001070 12582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:36.002157 12564 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:03:36.002274 12564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "0672a00262e644de94a6f08524c678e6"
format_stamp: "Formatted at 2026-08-12 03:03:35 on dist-test-slave-6bbx"
server_key: "c552a11161c6bbe7cfd60b0daa7b7a87"
server_key_iv: "d9267fb89ffdde79c665afd25cf37d9c"
server_key_version: "encryptionkey@0"
I20260812 03:03:36.002364 12564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:36.006646 12564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:36.007184 12564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:36.007340 12564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:36.013592 12564 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:38667
I20260812 03:03:36.013602 12667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:38667 every 8 connection(s)
I20260812 03:03:36.014417 12564 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:03:36.015877 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 12564
I20260812 03:03:36.015982 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:03:36.016227 5506 external_mini_cluster.cc:1428] Setting key ef788b3b4bec91cde5fc2127805150ad
I20260812 03:03:36.017923 12671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:36.021365 12674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:36.017503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55506 (local address 127.5.96.190:38667)
0812 03:03:36.018126 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:36.018141 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:36.018175 (+ 34us) server_negotiation.cc:407] Connection header received
0812 03:03:36.018784 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:36.018800 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:36.019179 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:36.019478 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:36.020225 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:36.020230 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:36.020240 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:03:36.020251 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:36.020302 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:03:36.020318 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:36.020440 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:36.020509 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 03:03:36.020734 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":123,"threads_started":1}
I20260812 03:03:36.023828 12671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0672a00262e644de94a6f08524c678e6: Bootstrap starting.
I20260812 03:03:36.026013 12671 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0672a00262e644de94a6f08524c678e6: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:36.026690 12671 log.cc:824] T 00000000000000000000000000000000 P 0672a00262e644de94a6f08524c678e6: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:36.028460 12671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0672a00262e644de94a6f08524c678e6: No bootstrap required, opened a new log
I20260812 03:03:36.031791 12671 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0672a00262e644de94a6f08524c678e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0672a00262e644de94a6f08524c678e6" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 38667 } }
I20260812 03:03:36.031991 12671 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0672a00262e644de94a6f08524c678e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:36.032052 12671 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0672a00262e644de94a6f08524c678e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0672a00262e644de94a6f08524c678e6, State: Initialized, Role: FOLLOWER
I20260812 03:03:36.032452 12671 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0672a00262e644de94a6f08524c678e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0672a00262e644de94a6f08524c678e6" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 38667 } }
I20260812 03:03:36.032578 12671 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0672a00262e644de94a6f08524c678e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:03:36.032640 12671 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0672a00262e644de94a6f08524c678e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:03:36.032740 12671 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0672a00262e644de94a6f08524c678e6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:36.033519 12671 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0672a00262e644de94a6f08524c678e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0672a00262e644de94a6f08524c678e6" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 38667 } }
I20260812 03:03:36.033850 12671 leader_election.cc:302] T 00000000000000000000000000000000 P 0672a00262e644de94a6f08524c678e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0672a00262e644de94a6f08524c678e6; no voters:
I20260812 03:03:36.034098 12671 leader_election.cc:288] T 00000000000000000000000000000000 P 0672a00262e644de94a6f08524c678e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:03:36.034195 12676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0672a00262e644de94a6f08524c678e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:36.034379 12676 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0672a00262e644de94a6f08524c678e6 [term 1 LEADER]: Becoming Leader. State: Replica: 0672a00262e644de94a6f08524c678e6, State: Running, Role: LEADER
I20260812 03:03:36.034655 12676 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0672a00262e644de94a6f08524c678e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0672a00262e644de94a6f08524c678e6" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 38667 } }
I20260812 03:03:36.035135 12671 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0672a00262e644de94a6f08524c678e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:03:36.036123 12681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0672a00262e644de94a6f08524c678e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0672a00262e644de94a6f08524c678e6. Latest consensus state: current_term: 1 leader_uuid: "0672a00262e644de94a6f08524c678e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0672a00262e644de94a6f08524c678e6" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 38667 } } }
I20260812 03:03:36.036262 12681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0672a00262e644de94a6f08524c678e6 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:36.036518 12679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0672a00262e644de94a6f08524c678e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0672a00262e644de94a6f08524c678e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0672a00262e644de94a6f08524c678e6" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 38667 } } }
I20260812 03:03:36.036621 12679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0672a00262e644de94a6f08524c678e6 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:36.036687 12691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:03:36.039829 12691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:03:36.045528 12691 catalog_manager.cc:1347] Generated new cluster ID: 59d3238d6f364d459ecb34e2d80bb786
I20260812 03:03:36.045583 12691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:03:36.054888 12691 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:03:36.055198 12691 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:03:36.069584 12691 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0672a00262e644de94a6f08524c678e6: Generated new TSK 0
I20260812 03:03:36.070164 12691 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:03:36.076858 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:0
--local_ip_for_outbound_sockets=127.5.96.129
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:38667
--builtin_ntp_servers=127.5.96.148:40961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 03:03:36.187463 12706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:36.187707 12706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:36.187745 12706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:36.191546 12706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:36.191588 12706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:36.191643 12706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:03:36.196185 12706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:38667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:36.197098 12706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:36.198364 12706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:36.207407 12721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:36.207576 12706 server_base.cc:1034] running on GCE node
W20260812 03:03:36.207396 12724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:36.207428 12718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:36.208105 12706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:36.208630 12706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:36.209787 12706 hybrid_clock.cc:648] HybridClock initialized: now 1786503816209774 us; error 28 us; skew 500 ppm
I20260812 03:03:36.211386 12706 webserver.cc:460] Webserver started at http://127.5.96.129:39571/ using document root <none> and password file <none>
I20260812 03:03:36.211948 12706 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:36.212015 12706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:36.212206 12706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:36.214133 12706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-0/data/instance:
uuid: "16d6a156d1ab412cbda9297906549479"
format_stamp: "Formatted at 2026-08-12 03:03:36 on dist-test-slave-6bbx"
server_key: "07ddef6bad6f1827f2259c678f65baec"
server_key_iv: "94ed233b2ec53bfd9687cbaf3ec24e75"
server_key_version: "encryptionkey@0"
I20260812 03:03:36.214641 12706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance:
uuid: "16d6a156d1ab412cbda9297906549479"
format_stamp: "Formatted at 2026-08-12 03:03:36 on dist-test-slave-6bbx"
server_key: "07ddef6bad6f1827f2259c678f65baec"
server_key_iv: "94ed233b2ec53bfd9687cbaf3ec24e75"
server_key_version: "encryptionkey@0"
I20260812 03:03:36.218017 12706 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 03:03:36.220296 12735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:36.221374 12706 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:03:36.221478 12706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "16d6a156d1ab412cbda9297906549479"
format_stamp: "Formatted at 2026-08-12 03:03:36 on dist-test-slave-6bbx"
server_key: "07ddef6bad6f1827f2259c678f65baec"
server_key_iv: "94ed233b2ec53bfd9687cbaf3ec24e75"
server_key_version: "encryptionkey@0"
I20260812 03:03:36.221565 12706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:36.225827 12706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:36.226346 12706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:36.226493 12706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:36.226992 12706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:36.227844 12706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:36.227892 12706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:36.227941 12706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:36.227970 12706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:36.235934 12706 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:37231
I20260812 03:03:36.235962 12911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:37231 every 8 connection(s)
I20260812 03:03:36.236726 12706 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:03:36.242056 12674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:36.238275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:39539 (local address 127.5.96.190:38667)
0812 03:03:36.238429 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:36.238434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:36.239371 (+ 937us) server_negotiation.cc:407] Connection header received
0812 03:03:36.240703 (+ 1332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:36.240706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:36.240743 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:36.240818 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:36.241745 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:36.241756 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:36.241757 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:03:36.241765 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:36.241777 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:03:36.241784 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:36.241839 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:36.241922 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 03:03:36.241953 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:03:36.242808 12917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:36.238646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:38667 (local address 127.5.96.129:39539)
0812 03:03:36.239243 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:36.239280 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:36.240519 (+ 1239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:36.240948 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:36.240955 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:36.241035 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:36.241538 (+ 503us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:03:36.241540 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:36.241576 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:03:36.241639 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:36.241834 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:36.241838 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:36.241841 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:03:36.241951 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 03:03:36.242136 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":110,"threads_started":1}
I20260812 03:03:36.243361 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 12706
I20260812 03:03:36.243472 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance
I20260812 03:03:36.243800 5506 external_mini_cluster.cc:1428] Setting key 2df7c5418745320dd80fb64da54f90c6
I20260812 03:03:36.244012 12912 heartbeater.cc:347] Connected to a master server at 127.5.96.190:38667
I20260812 03:03:36.244218 12912 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:36.244589 12912 heartbeater.cc:511] Master 127.5.96.190:38667 requested a full tablet report, sending...
I20260812 03:03:36.245617 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:0
--local_ip_for_outbound_sockets=127.5.96.130
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:38667
--builtin_ntp_servers=127.5.96.148:40961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 03:03:36.246091 12606 ts_manager.cc:194] Registered new tserver with Master: 16d6a156d1ab412cbda9297906549479 (127.5.96.129:37231)
W20260812 03:03:36.352005 12919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:36.352221 12919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:36.352263 12919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:36.356124 12919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:36.356166 12919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:36.356217 12919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:03:36.360656 12919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:38667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:36.361539 12919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:36.362746 12919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:36.371520 12928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:36.371527 12926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:36.371526 12924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:36.371712 12919 server_base.cc:1034] running on GCE node
I20260812 03:03:36.372197 12919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:36.372721 12919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:36.373880 12919 hybrid_clock.cc:648] HybridClock initialized: now 1786503816373853 us; error 38 us; skew 500 ppm
I20260812 03:03:36.375464 12919 webserver.cc:460] Webserver started at http://127.5.96.130:43237/ using document root <none> and password file <none>
I20260812 03:03:36.376015 12919 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:36.376076 12919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:36.376266 12919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:36.378098 12919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-1/data/instance:
uuid: "02445de2e32943639c33bba219047e83"
format_stamp: "Formatted at 2026-08-12 03:03:36 on dist-test-slave-6bbx"
server_key: "3b5a5802102e001b598bd8f4a08634ed"
server_key_iv: "da201b912a5f2af8dd79d006c6587ba2"
server_key_version: "encryptionkey@0"
I20260812 03:03:36.378638 12919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance:
uuid: "02445de2e32943639c33bba219047e83"
format_stamp: "Formatted at 2026-08-12 03:03:36 on dist-test-slave-6bbx"
server_key: "3b5a5802102e001b598bd8f4a08634ed"
server_key_iv: "da201b912a5f2af8dd79d006c6587ba2"
server_key_version: "encryptionkey@0"
I20260812 03:03:36.382138 12919 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 03:03:36.384284 12937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:36.385452 12919 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:36.385545 12919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "02445de2e32943639c33bba219047e83"
format_stamp: "Formatted at 2026-08-12 03:03:36 on dist-test-slave-6bbx"
server_key: "3b5a5802102e001b598bd8f4a08634ed"
server_key_iv: "da201b912a5f2af8dd79d006c6587ba2"
server_key_version: "encryptionkey@0"
I20260812 03:03:36.385638 12919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:36.389791 12919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:36.390308 12919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:36.390450 12919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:36.390934 12919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:36.391799 12919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:36.391855 12919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:36.391909 12919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:36.391938 12919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:36.399451 12919 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:33469
I20260812 03:03:36.399483 13123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:33469 every 8 connection(s)
I20260812 03:03:36.400254 12919 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:03:36.401949 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 12919
I20260812 03:03:36.402068 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance
I20260812 03:03:36.402380 5506 external_mini_cluster.cc:1428] Setting key 117072283a042a3173a1f2de8aac1ec7
I20260812 03:03:36.404381 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:0
--local_ip_for_outbound_sockets=127.5.96.131
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:38667
--builtin_ntp_servers=127.5.96.148:40961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 03:03:36.406034 12674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:36.401766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:48263 (local address 127.5.96.190:38667)
0812 03:03:36.401884 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:36.401887 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:36.403070 (+ 1183us) server_negotiation.cc:407] Connection header received
0812 03:03:36.404478 (+ 1408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:36.404481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:36.404521 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:36.404595 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:36.405614 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:36.405618 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:36.405620 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:03:36.405632 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:36.405653 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:03:36.405662 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:36.405732 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:36.405869 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 03:03:36.405909 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:03:36.406879 13127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:36.402207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:38667 (local address 127.5.96.130:48263)
0812 03:03:36.402937 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:36.402986 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:36.404291 (+ 1305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:36.404759 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:36.404766 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:36.404845 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:36.405358 (+ 513us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:03:36.405359 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:36.405417 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:03:36.405491 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:36.405727 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:36.405735 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:36.405741 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:03:36.405901 (+ 160us) client_negotiation.cc:240] Negotiation successful
0812 03:03:36.406117 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":141,"threads_started":1}
I20260812 03:03:36.407904 13124 heartbeater.cc:347] Connected to a master server at 127.5.96.190:38667
I20260812 03:03:36.408104 13124 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:36.408454 13124 heartbeater.cc:511] Master 127.5.96.190:38667 requested a full tablet report, sending...
I20260812 03:03:36.409365 12606 ts_manager.cc:194] Registered new tserver with Master: 02445de2e32943639c33bba219047e83 (127.5.96.130:33469)
W20260812 03:03:36.513337 13128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:36.513559 13128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:36.513600 13128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:36.517423 13128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:36.517467 13128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:36.517530 13128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:03:36.522149 13128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:38667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:36.523063 13128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:36.524374 13128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:36.533254 13137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:36.533274 13141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:36.533265 13134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:36.533378 13128 server_base.cc:1034] running on GCE node
I20260812 03:03:36.533849 13128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:36.534392 13128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:36.535549 13128 hybrid_clock.cc:648] HybridClock initialized: now 1786503816535522 us; error 26 us; skew 500 ppm
I20260812 03:03:36.537344 13128 webserver.cc:460] Webserver started at http://127.5.96.131:39299/ using document root <none> and password file <none>
I20260812 03:03:36.537847 13128 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:36.537905 13128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:36.538090 13128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:36.539983 13128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-2/data/instance:
uuid: "789502111bfd4cd883f432188d4de7e6"
format_stamp: "Formatted at 2026-08-12 03:03:36 on dist-test-slave-6bbx"
server_key: "3f08f02d8eae25d58c65e5bb07fa2a7b"
server_key_iv: "9ac74c4aec206509e7c8dcb9fce05fe7"
server_key_version: "encryptionkey@0"
I20260812 03:03:36.540495 13128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance:
uuid: "789502111bfd4cd883f432188d4de7e6"
format_stamp: "Formatted at 2026-08-12 03:03:36 on dist-test-slave-6bbx"
server_key: "3f08f02d8eae25d58c65e5bb07fa2a7b"
server_key_iv: "9ac74c4aec206509e7c8dcb9fce05fe7"
server_key_version: "encryptionkey@0"
I20260812 03:03:36.543838 13128 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:03:36.545955 13165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:36.547024 13128 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:36.547118 13128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "789502111bfd4cd883f432188d4de7e6"
format_stamp: "Formatted at 2026-08-12 03:03:36 on dist-test-slave-6bbx"
server_key: "3f08f02d8eae25d58c65e5bb07fa2a7b"
server_key_iv: "9ac74c4aec206509e7c8dcb9fce05fe7"
server_key_version: "encryptionkey@0"
I20260812 03:03:36.547211 13128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:36.551388 13128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:36.551970 13128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:36.552117 13128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:36.552608 13128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:36.553423 13128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:36.553468 13128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:36.553507 13128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:36.553537 13128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:36.561432 13128 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:44001
I20260812 03:03:36.561455 13348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:44001 every 8 connection(s)
I20260812 03:03:36.562207 13128 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:03:36.567687 12674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:36.563856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:48977 (local address 127.5.96.190:38667)
0812 03:03:36.563966 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:36.563969 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:36.564937 (+ 968us) server_negotiation.cc:407] Connection header received
0812 03:03:36.566130 (+ 1193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:36.566134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:36.566182 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:36.566260 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:36.567241 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:36.567244 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:36.567255 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:03:36.567269 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:36.567289 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:03:36.567297 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:36.567371 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:36.567491 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 03:03:36.567534 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 03:03:36.568593 13351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:36.564204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:38667 (local address 127.5.96.131:48977)
0812 03:03:36.564810 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:36.564846 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:36.565936 (+ 1090us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:36.566410 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:36.566417 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:36.566499 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:36.567001 (+ 502us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:03:36.567002 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:36.567040 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:03:36.567107 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:36.567367 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:36.567376 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:36.567382 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:03:36.567527 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 03:03:36.567783 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":98,"threads_started":1}
I20260812 03:03:36.569620 13349 heartbeater.cc:347] Connected to a master server at 127.5.96.190:38667
I20260812 03:03:36.569819 13349 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:36.570163 13349 heartbeater.cc:511] Master 127.5.96.190:38667 requested a full tablet report, sending...
I20260812 03:03:36.571103 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 13128
I20260812 03:03:36.571106 12606 ts_manager.cc:194] Registered new tserver with Master: 789502111bfd4cd883f432188d4de7e6 (127.5.96.131:44001)
I20260812 03:03:36.571214 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance
I20260812 03:03:36.571502 5506 external_mini_cluster.cc:1428] Setting key 1522da07a4840fffa64fcf912dd00051
I20260812 03:03:36.573278 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:03:36.576506 12674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:36.575035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55508 (local address 127.5.96.190:38667)
0812 03:03:36.575155 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:36.575159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:36.575312 (+ 153us) server_negotiation.cc:407] Connection header received
0812 03:03:36.575413 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:36.575417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:36.575446 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:36.575508 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:36.576168 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:36.576172 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:36.576174 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:03:36.576185 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:36.576204 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:03:36.576212 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:36.576288 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:36.576360 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 03:03:36.576397 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:03:36.578792 12606 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55508:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:03:36.581681 12606 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:03:36.591593 13366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:36.588805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:44001 (local address 127.0.0.1:43440)
0812 03:03:36.589176 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:36.589221 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:36.589353 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:36.590201 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:36.590207 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:36.590292 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:36.590848 (+ 556us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:03:36.590850 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:36.590871 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:03:36.590937 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:36.591310 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:36.591319 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:36.591324 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:03:36.591433 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 03:03:36.591495 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":79,"threads_started":1}
I20260812 03:03:36.591921 13367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:36.588862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43440 (local address 127.5.96.131:44001)
0812 03:03:36.589251 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:36.589263 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:36.589355 (+ 92us) server_negotiation.cc:407] Connection header received
0812 03:03:36.589486 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:36.589493 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:36.589672 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:36.589805 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:36.591063 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:36.591069 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:36.591080 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:03:36.591092 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:36.591133 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:03:36.591161 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:36.591265 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:36.591459 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 03:03:36.591531 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":156,"threads_started":1}
I20260812 03:03:36.592175 13372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:36.589354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:37231 (local address 127.0.0.1:38298)
0812 03:03:36.589757 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:36.589769 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:36.589850 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:36.590727 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:36.590730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:36.590742 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:36.591258 (+ 516us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:03:36.591260 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:36.591271 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:03:36.591297 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:36.591942 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:36.591944 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:36.591947 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:03:36.592029 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 03:03:36.592064 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":93,"threads_started":1}
I20260812 03:03:36.592468 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:36.589654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38298 (local address 127.5.96.129:37231)
0812 03:03:36.590098 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:36.590110 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:36.590132 (+ 22us) server_negotiation.cc:407] Connection header received
0812 03:03:36.590185 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:36.590190 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:36.590356 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:36.591122 (+ 766us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:36.591758 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:36.591764 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:36.591775 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:03:36.591787 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:36.591827 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:03:36.591862 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:36.591971 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:36.592231 (+ 260us) server_negotiation.cc:299] Negotiation successful
0812 03:03:36.592307 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":128,"threads_started":1}
I20260812 03:03:36.592700 13368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:36.589462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:33469 (local address 127.0.0.1:55782)
0812 03:03:36.590342 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:36.590354 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:36.590434 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:36.590908 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:36.590911 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:36.590924 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:36.592057 (+ 1133us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:03:36.592059 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:36.592070 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:03:36.592096 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:36.592470 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:36.592473 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:36.592476 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:03:36.592555 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 03:03:36.592588 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827,"thread_start_us":89,"threads_started":1}
I20260812 03:03:36.592824 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:36.589736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55782 (local address 127.5.96.130:33469)
0812 03:03:36.590139 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:36.590153 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:36.590527 (+ 374us) server_negotiation.cc:407] Connection header received
0812 03:03:36.590588 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:36.590593 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:36.590765 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:36.590904 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:36.592216 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:36.592230 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:36.592241 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:03:36.592253 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:36.592295 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:03:36.592330 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:36.592426 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:36.592597 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 03:03:36.592666 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":130,"threads_started":1}
I20260812 03:03:36.594851 13244 tablet_service.cc:1467] Processing CreateTablet for tablet 4e1d7fc626834fcd87b292bcff21a73c (DEFAULT_TABLE table=test-table [id=a84ae03a9a80409bb259f6d599597890]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:36.595077 12810 tablet_service.cc:1467] Processing CreateTablet for tablet 4e1d7fc626834fcd87b292bcff21a73c (DEFAULT_TABLE table=test-table [id=a84ae03a9a80409bb259f6d599597890]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:36.595445 13015 tablet_service.cc:1467] Processing CreateTablet for tablet 4e1d7fc626834fcd87b292bcff21a73c (DEFAULT_TABLE table=test-table [id=a84ae03a9a80409bb259f6d599597890]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:36.595963 13244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e1d7fc626834fcd87b292bcff21a73c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:36.595971 12810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e1d7fc626834fcd87b292bcff21a73c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:36.596379 13015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e1d7fc626834fcd87b292bcff21a73c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:36.601943 13376 tablet_bootstrap.cc:492] T 4e1d7fc626834fcd87b292bcff21a73c P 16d6a156d1ab412cbda9297906549479: Bootstrap starting.
I20260812 03:03:36.603271 13379 tablet_bootstrap.cc:492] T 4e1d7fc626834fcd87b292bcff21a73c P 02445de2e32943639c33bba219047e83: Bootstrap starting.
I20260812 03:03:36.604296 13376 tablet_bootstrap.cc:654] T 4e1d7fc626834fcd87b292bcff21a73c P 16d6a156d1ab412cbda9297906549479: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:36.604959 13379 tablet_bootstrap.cc:654] T 4e1d7fc626834fcd87b292bcff21a73c P 02445de2e32943639c33bba219047e83: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:36.605021 13376 log.cc:824] T 4e1d7fc626834fcd87b292bcff21a73c P 16d6a156d1ab412cbda9297906549479: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:36.605716 13379 log.cc:824] T 4e1d7fc626834fcd87b292bcff21a73c P 02445de2e32943639c33bba219047e83: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:36.606982 13376 tablet_bootstrap.cc:492] T 4e1d7fc626834fcd87b292bcff21a73c P 16d6a156d1ab412cbda9297906549479: No bootstrap required, opened a new log
I20260812 03:03:36.607151 13376 ts_tablet_manager.cc:1397] T 4e1d7fc626834fcd87b292bcff21a73c P 16d6a156d1ab412cbda9297906549479: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 03:03:36.607365 13379 tablet_bootstrap.cc:492] T 4e1d7fc626834fcd87b292bcff21a73c P 02445de2e32943639c33bba219047e83: No bootstrap required, opened a new log
I20260812 03:03:36.607528 13379 ts_tablet_manager.cc:1397] T 4e1d7fc626834fcd87b292bcff21a73c P 02445de2e32943639c33bba219047e83: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260812 03:03:36.607944 13380 tablet_bootstrap.cc:492] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6: Bootstrap starting.
I20260812 03:03:36.609993 13380 tablet_bootstrap.cc:654] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:36.610666 13379 raft_consensus.cc:348] T 4e1d7fc626834fcd87b292bcff21a73c P 02445de2e32943639c33bba219047e83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "789502111bfd4cd883f432188d4de7e6" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44001 } } peers { permanent_uuid: "02445de2e32943639c33bba219047e83" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33469 } } peers { permanent_uuid: "16d6a156d1ab412cbda9297906549479" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 37231 } }
I20260812 03:03:36.610780 13380 log.cc:824] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:36.610874 13379 raft_consensus.cc:374] T 4e1d7fc626834fcd87b292bcff21a73c P 02445de2e32943639c33bba219047e83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:36.610929 13379 raft_consensus.cc:729] T 4e1d7fc626834fcd87b292bcff21a73c P 02445de2e32943639c33bba219047e83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02445de2e32943639c33bba219047e83, State: Initialized, Role: FOLLOWER
I20260812 03:03:36.611387 13379 consensus_queue.cc:260] T 4e1d7fc626834fcd87b292bcff21a73c P 02445de2e32943639c33bba219047e83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "789502111bfd4cd883f432188d4de7e6" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44001 } } peers { permanent_uuid: "02445de2e32943639c33bba219047e83" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33469 } } peers { permanent_uuid: "16d6a156d1ab412cbda9297906549479" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 37231 } }
I20260812 03:03:36.611318 13376 raft_consensus.cc:348] T 4e1d7fc626834fcd87b292bcff21a73c P 16d6a156d1ab412cbda9297906549479 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "789502111bfd4cd883f432188d4de7e6" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44001 } } peers { permanent_uuid: "02445de2e32943639c33bba219047e83" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33469 } } peers { permanent_uuid: "16d6a156d1ab412cbda9297906549479" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 37231 } }
I20260812 03:03:36.611549 13376 raft_consensus.cc:374] T 4e1d7fc626834fcd87b292bcff21a73c P 16d6a156d1ab412cbda9297906549479 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:36.611610 13376 raft_consensus.cc:729] T 4e1d7fc626834fcd87b292bcff21a73c P 16d6a156d1ab412cbda9297906549479 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16d6a156d1ab412cbda9297906549479, State: Initialized, Role: FOLLOWER
I20260812 03:03:36.612043 13376 consensus_queue.cc:260] T 4e1d7fc626834fcd87b292bcff21a73c P 16d6a156d1ab412cbda9297906549479 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "789502111bfd4cd883f432188d4de7e6" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44001 } } peers { permanent_uuid: "02445de2e32943639c33bba219047e83" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33469 } } peers { permanent_uuid: "16d6a156d1ab412cbda9297906549479" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 37231 } }
I20260812 03:03:36.612267 13124 heartbeater.cc:503] Master 127.5.96.190:38667 was elected leader, sending a full tablet report...
I20260812 03:03:36.612962 13376 ts_tablet_manager.cc:1428] T 4e1d7fc626834fcd87b292bcff21a73c P 16d6a156d1ab412cbda9297906549479: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 03:03:36.613127 13380 tablet_bootstrap.cc:492] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6: No bootstrap required, opened a new log
I20260812 03:03:36.613247 13379 ts_tablet_manager.cc:1428] T 4e1d7fc626834fcd87b292bcff21a73c P 02445de2e32943639c33bba219047e83: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260812 03:03:36.613271 12912 heartbeater.cc:503] Master 127.5.96.190:38667 was elected leader, sending a full tablet report...
I20260812 03:03:36.613327 13380 ts_tablet_manager.cc:1397] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:03:36.617084 13380 raft_consensus.cc:348] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "789502111bfd4cd883f432188d4de7e6" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44001 } } peers { permanent_uuid: "02445de2e32943639c33bba219047e83" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33469 } } peers { permanent_uuid: "16d6a156d1ab412cbda9297906549479" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 37231 } }
I20260812 03:03:36.617277 13380 raft_consensus.cc:374] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:36.617331 13380 raft_consensus.cc:729] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 789502111bfd4cd883f432188d4de7e6, State: Initialized, Role: FOLLOWER
I20260812 03:03:36.617697 13380 consensus_queue.cc:260] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "789502111bfd4cd883f432188d4de7e6" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44001 } } peers { permanent_uuid: "02445de2e32943639c33bba219047e83" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33469 } } peers { permanent_uuid: "16d6a156d1ab412cbda9297906549479" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 37231 } }
I20260812 03:03:36.618268 13349 heartbeater.cc:503] Master 127.5.96.190:38667 was elected leader, sending a full tablet report...
I20260812 03:03:36.618527 13380 ts_tablet_manager.cc:1428] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260812 03:03:36.651067 13126 tablet.cc:2369] T 4e1d7fc626834fcd87b292bcff21a73c P 02445de2e32943639c33bba219047e83: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:03:36.737697 12913 tablet.cc:2369] T 4e1d7fc626834fcd87b292bcff21a73c P 16d6a156d1ab412cbda9297906549479: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:03:36.813190 13350 tablet.cc:2369] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:03:36.845347 13391 raft_consensus.cc:482] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:03:36.845534 13391 raft_consensus.cc:504] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "789502111bfd4cd883f432188d4de7e6" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44001 } } peers { permanent_uuid: "02445de2e32943639c33bba219047e83" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33469 } } peers { permanent_uuid: "16d6a156d1ab412cbda9297906549479" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 37231 } }
I20260812 03:03:36.846612 13391 leader_election.cc:288] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02445de2e32943639c33bba219047e83 (127.5.96.130:33469), 16d6a156d1ab412cbda9297906549479 (127.5.96.129:37231)
I20260812 03:03:36.848441 13351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:36.846827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:33469 (local address 127.5.96.131:46455)
0812 03:03:36.846982 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:36.846997 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:36.847081 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:36.847389 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:36.847392 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:36.847408 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:36.847885 (+ 477us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:03:36.847887 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:36.847901 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:03:36.847932 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:36.848187 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:36.848191 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:36.848194 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:03:36.848277 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 03:03:36.848314 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 03:03:36.848455 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:36.846948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:46455 (local address 127.5.96.130:33469)
0812 03:03:36.847190 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:36.847194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:36.847204 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:03:36.847251 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:36.847253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:36.847284 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:36.847350 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:36.848060 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:36.848064 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:36.848067 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:03:36.848079 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:36.848103 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:03:36.848114 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:36.848195 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:36.848283 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 03:03:36.848327 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166}
I20260812 03:03:36.848538 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:36.846953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:35459 (local address 127.5.96.129:37231)
0812 03:03:36.847092 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:36.847096 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:36.847364 (+ 268us) server_negotiation.cc:407] Connection header received
0812 03:03:36.847454 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:36.847457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:36.847493 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:36.847558 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:36.848161 (+ 603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:36.848165 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:36.848167 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:03:36.848181 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:36.848204 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:03:36.848214 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:36.848309 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:36.848388 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 03:03:36.848425 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 03:03:36.848574 13393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:36.846839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:37231 (local address 127.5.96.131:35459)
0812 03:03:36.847270 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:36.847282 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:36.847374 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:36.847590 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:36.847593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:36.847608 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:36.848000 (+ 392us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:03:36.848002 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:36.848013 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:03:36.848041 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:36.848304 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:36.848307 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:36.848310 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:03:36.848379 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 03:03:36.848412 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":104,"thread_start_us":105,"threads_started":1}
I20260812 03:03:36.849238 12839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e1d7fc626834fcd87b292bcff21a73c" candidate_uuid: "789502111bfd4cd883f432188d4de7e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16d6a156d1ab412cbda9297906549479" is_pre_election: true
I20260812 03:03:36.849228 13043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e1d7fc626834fcd87b292bcff21a73c" candidate_uuid: "789502111bfd4cd883f432188d4de7e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02445de2e32943639c33bba219047e83" is_pre_election: true
I20260812 03:03:36.849534 13043 raft_consensus.cc:2454] T 4e1d7fc626834fcd87b292bcff21a73c P 02445de2e32943639c33bba219047e83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 789502111bfd4cd883f432188d4de7e6 in term 0.
I20260812 03:03:36.849534 12839 raft_consensus.cc:2454] T 4e1d7fc626834fcd87b292bcff21a73c P 16d6a156d1ab412cbda9297906549479 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 789502111bfd4cd883f432188d4de7e6 in term 0.
I20260812 03:03:36.849997 13166 leader_election.cc:302] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16d6a156d1ab412cbda9297906549479, 789502111bfd4cd883f432188d4de7e6; no voters:
I20260812 03:03:36.850274 13391 raft_consensus.cc:2781] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:03:36.850351 13391 raft_consensus.cc:482] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:03:36.850394 13391 raft_consensus.cc:3037] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:36.851488 13391 raft_consensus.cc:504] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "789502111bfd4cd883f432188d4de7e6" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44001 } } peers { permanent_uuid: "02445de2e32943639c33bba219047e83" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33469 } } peers { permanent_uuid: "16d6a156d1ab412cbda9297906549479" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 37231 } }
I20260812 03:03:36.851827 13391 leader_election.cc:288] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6 [CANDIDATE]: Term 1 election: Requested vote from peers 02445de2e32943639c33bba219047e83 (127.5.96.130:33469), 16d6a156d1ab412cbda9297906549479 (127.5.96.129:37231)
I20260812 03:03:36.852133 13043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e1d7fc626834fcd87b292bcff21a73c" candidate_uuid: "789502111bfd4cd883f432188d4de7e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02445de2e32943639c33bba219047e83"
I20260812 03:03:36.852250 13043 raft_consensus.cc:3037] T 4e1d7fc626834fcd87b292bcff21a73c P 02445de2e32943639c33bba219047e83 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:36.853271 13043 raft_consensus.cc:2454] T 4e1d7fc626834fcd87b292bcff21a73c P 02445de2e32943639c33bba219047e83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 789502111bfd4cd883f432188d4de7e6 in term 1.
I20260812 03:03:36.853521 12839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e1d7fc626834fcd87b292bcff21a73c" candidate_uuid: "789502111bfd4cd883f432188d4de7e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16d6a156d1ab412cbda9297906549479"
I20260812 03:03:36.853578 13171 leader_election.cc:302] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02445de2e32943639c33bba219047e83, 789502111bfd4cd883f432188d4de7e6; no voters:
I20260812 03:03:36.853675 12839 raft_consensus.cc:3037] T 4e1d7fc626834fcd87b292bcff21a73c P 16d6a156d1ab412cbda9297906549479 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:36.853730 13391 raft_consensus.cc:2781] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:36.853931 13391 raft_consensus.cc:686] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6 [term 1 LEADER]: Becoming Leader. State: Replica: 789502111bfd4cd883f432188d4de7e6, State: Running, Role: LEADER
I20260812 03:03:36.854246 13391 consensus_queue.cc:237] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "789502111bfd4cd883f432188d4de7e6" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44001 } } peers { permanent_uuid: "02445de2e32943639c33bba219047e83" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33469 } } peers { permanent_uuid: "16d6a156d1ab412cbda9297906549479" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 37231 } }
I20260812 03:03:36.854802 12839 raft_consensus.cc:2454] T 4e1d7fc626834fcd87b292bcff21a73c P 16d6a156d1ab412cbda9297906549479 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 789502111bfd4cd883f432188d4de7e6 in term 1.
I20260812 03:03:36.857429 12606 catalog_manager.cc:5495] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 789502111bfd4cd883f432188d4de7e6 (127.5.96.131). New cstate: current_term: 1 leader_uuid: "789502111bfd4cd883f432188d4de7e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "789502111bfd4cd883f432188d4de7e6" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02445de2e32943639c33bba219047e83" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16d6a156d1ab412cbda9297906549479" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 37231 } health_report { overall_health: UNKNOWN } } }
I20260812 03:03:36.879551 13367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:36.878408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43444 (local address 127.5.96.131:44001)
0812 03:03:36.878520 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:36.878524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:36.878536 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:03:36.878637 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:36.878639 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:36.878694 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:36.878756 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:36.879288 (+ 532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:36.879290 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:36.879293 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:03:36.879303 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:36.879321 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:03:36.879329 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:36.879393 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:36.879428 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 03:03:36.879458 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:03:36.885161 13043 raft_consensus.cc:1261] T 4e1d7fc626834fcd87b292bcff21a73c P 02445de2e32943639c33bba219047e83 [term 1 FOLLOWER]: Refusing update from remote peer 789502111bfd4cd883f432188d4de7e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:36.885162 12839 raft_consensus.cc:1261] T 4e1d7fc626834fcd87b292bcff21a73c P 16d6a156d1ab412cbda9297906549479 [term 1 FOLLOWER]: Refusing update from remote peer 789502111bfd4cd883f432188d4de7e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:36.885797 13391 consensus_queue.cc:1035] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02445de2e32943639c33bba219047e83" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:36.885972 13397 consensus_queue.cc:1035] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16d6a156d1ab412cbda9297906549479" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 37231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:36.893399 13405 mvcc.cc:205] Tried to move back new op lower bound from 7317519633954689024 to 7317519633837367296. Current Snapshot: MvccSnapshot[applied={T|T < 7317519633954689024}]
I20260812 03:03:36.901847 12606 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:36.901976 12606 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:36.904162 12606 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0672a00262e644de94a6f08524c678e6: Sending DeleteTablet for 3 replicas of tablet 4e1d7fc626834fcd87b292bcff21a73c
I20260812 03:03:36.904920 13015 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e1d7fc626834fcd87b292bcff21a73c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:36 UTC) from {username='slave'} at 127.0.0.1:55782
I20260812 03:03:36.905332 13414 tablet_replica.cc:330] T 4e1d7fc626834fcd87b292bcff21a73c P 02445de2e32943639c33bba219047e83: stopping tablet replica
I20260812 03:03:36.905370 12810 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e1d7fc626834fcd87b292bcff21a73c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:36 UTC) from {username='slave'} at 127.0.0.1:38298
I20260812 03:03:36.905620 13414 raft_consensus.cc:2229] T 4e1d7fc626834fcd87b292bcff21a73c P 02445de2e32943639c33bba219047e83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:36.905936 13415 tablet_replica.cc:330] T 4e1d7fc626834fcd87b292bcff21a73c P 16d6a156d1ab412cbda9297906549479: stopping tablet replica
I20260812 03:03:36.905961 13414 raft_consensus.cc:2258] T 4e1d7fc626834fcd87b292bcff21a73c P 02445de2e32943639c33bba219047e83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:36.906193 13244 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e1d7fc626834fcd87b292bcff21a73c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:36 UTC) from {username='slave'} at 127.0.0.1:43440
I20260812 03:03:36.906337 13415 raft_consensus.cc:2229] T 4e1d7fc626834fcd87b292bcff21a73c P 16d6a156d1ab412cbda9297906549479 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:36.906746 13415 raft_consensus.cc:2258] T 4e1d7fc626834fcd87b292bcff21a73c P 16d6a156d1ab412cbda9297906549479 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:36.906870 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 12706
I20260812 03:03:36.907723 13414 ts_tablet_manager.cc:1905] T 4e1d7fc626834fcd87b292bcff21a73c P 02445de2e32943639c33bba219047e83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:36.910799 13414 ts_tablet_manager.cc:1918] T 4e1d7fc626834fcd87b292bcff21a73c P 02445de2e32943639c33bba219047e83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:03:36.910918 13414 log.cc:1196] T 4e1d7fc626834fcd87b292bcff21a73c P 02445de2e32943639c33bba219047e83: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-1/wal/wals/4e1d7fc626834fcd87b292bcff21a73c
I20260812 03:03:36.911144 13416 tablet_replica.cc:330] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6: stopping tablet replica
I20260812 03:03:36.911283 13414 ts_tablet_manager.cc:1939] T 4e1d7fc626834fcd87b292bcff21a73c P 02445de2e32943639c33bba219047e83: Deleting consensus metadata
I20260812 03:03:36.911520 13416 raft_consensus.cc:2229] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:03:36.911934 13416 raft_consensus.cc:2258] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:36.913830 13416 ts_tablet_manager.cc:1905] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:36.916224 13416 ts_tablet_manager.cc:1918] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:03:36.916322 13416 log.cc:1196] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786503790637001-5506-0/minicluster-data/ts-2/wal/wals/4e1d7fc626834fcd87b292bcff21a73c
I20260812 03:03:36.916627 13416 ts_tablet_manager.cc:1939] T 4e1d7fc626834fcd87b292bcff21a73c P 789502111bfd4cd883f432188d4de7e6: Deleting consensus metadata
I20260812 03:03:36.918540 12587 catalog_manager.cc:4841] TS 789502111bfd4cd883f432188d4de7e6 (127.5.96.131:44001): tablet 4e1d7fc626834fcd87b292bcff21a73c (table test-table [id=a84ae03a9a80409bb259f6d599597890]) successfully deleted
I20260812 03:03:36.918725 12587 catalog_manager.cc:4841] TS 02445de2e32943639c33bba219047e83 (127.5.96.130:33469): tablet 4e1d7fc626834fcd87b292bcff21a73c (table test-table [id=a84ae03a9a80409bb259f6d599597890]) successfully deleted
W20260812 03:03:36.925191 12584 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.96.129:37231 (error 108)
I20260812 03:03:36.925906 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 12919
I20260812 03:03:36.926188 13368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:36.925908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:37231 (local address 127.0.0.1:38310)
0812 03:03:36.926031 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:36.926104 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.129:37231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 03:03:36.928452 12584 catalog_manager.cc:4584] TS 16d6a156d1ab412cbda9297906549479 (127.5.96.129:37231): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4e1d7fc626834fcd87b292bcff21a73c: Network error: Client connection negotiation failed: client connection to 127.5.96.129:37231: connect: Connection refused (error 111)
I20260812 03:03:36.939821 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 13128
I20260812 03:03:36.949942 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 12564
2026-08-12T03:03:36Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1120 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 03:03:36.974378 5506 mini_oidc.cc:152] Starting JWKS server
I20260812 03:03:36.976222 5506 webserver.cc:460] Webserver started at https://127.0.0.1:40235/ using document root <none> and password file <none>
I20260812 03:03:36.976389 5506 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 03:03:36.976632 5506 webserver.cc:460] Webserver started at http://127.0.0.1:38583/ using document root <none> and password file <none>
2026-08-12T03:03:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:03:36Z Disabled control of system clock
I20260812 03:03:36.985528 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:35095
--webserver_interface=127.5.96.190
--webserver_port=0
--builtin_ntp_servers=127.5.96.148:43153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:35095
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40235/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786503790637001-5506-0/testchainca.cert with env {}
W20260812 03:03:37.093214 13428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:37.093436 13428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:37.093472 13428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:37.097326 13428 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:03:37.097373 13428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:37.097386 13428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:37.097406 13428 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:03:37.097420 13428 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 03:03:37.097438 13428 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 03:03:37.097451 13428 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40235/jwks.json
I20260812 03:03:37.102495 13428 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:43153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:35095
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:35095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40235/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786503790637001-5506-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:37.103353 13428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:37.104249 13428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:37.112403 13439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:37.112429 13437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:37.112430 13442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:37.112579 13428 server_base.cc:1034] running on GCE node
I20260812 03:03:37.113247 13428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:37.114235 13428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:37.115396 13428 hybrid_clock.cc:648] HybridClock initialized: now 1786503817115367 us; error 36 us; skew 500 ppm
I20260812 03:03:37.117089 13428 webserver.cc:460] Webserver started at http://127.5.96.190:35365/ using document root <none> and password file <none>
I20260812 03:03:37.117619 13428 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:37.117681 13428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:37.117865 13428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:37.119740 13428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "d27836aac8654660ac69f6ca1a1a6ffd"
format_stamp: "Formatted at 2026-08-12 03:03:37 on dist-test-slave-6bbx"
server_key: "b8d3a348563c47b812f11d28b90eafaf"
server_key_iv: "64986af074e8a1606f8f9cbde0b21b21"
server_key_version: "encryptionkey@0"
I20260812 03:03:37.120262 13428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "d27836aac8654660ac69f6ca1a1a6ffd"
format_stamp: "Formatted at 2026-08-12 03:03:37 on dist-test-slave-6bbx"
server_key: "b8d3a348563c47b812f11d28b90eafaf"
server_key_iv: "64986af074e8a1606f8f9cbde0b21b21"
server_key_version: "encryptionkey@0"
I20260812 03:03:37.123703 13428 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260812 03:03:37.125931 13449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:37.127053 13428 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:37.127166 13428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "d27836aac8654660ac69f6ca1a1a6ffd"
format_stamp: "Formatted at 2026-08-12 03:03:37 on dist-test-slave-6bbx"
server_key: "b8d3a348563c47b812f11d28b90eafaf"
server_key_iv: "64986af074e8a1606f8f9cbde0b21b21"
server_key_version: "encryptionkey@0"
I20260812 03:03:37.127269 13428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:37.145117 13428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:37.145682 13428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:37.145833 13428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:37.152271 13428 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:35095
I20260812 03:03:37.152290 13539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:35095 every 8 connection(s)
I20260812 03:03:37.153090 13428 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:03:37.155741 13541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:37.160522 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 13428
I20260812 03:03:37.160626 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:03:37.160876 5506 external_mini_cluster.cc:1428] Setting key 92f989627c166d9238db370293248585
I20260812 03:03:37.160878 13541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d27836aac8654660ac69f6ca1a1a6ffd: Bootstrap starting.
I20260812 03:03:37.163410 13541 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d27836aac8654660ac69f6ca1a1a6ffd: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:37.164285 13541 log.cc:824] T 00000000000000000000000000000000 P d27836aac8654660ac69f6ca1a1a6ffd: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:37.166304 13541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d27836aac8654660ac69f6ca1a1a6ffd: No bootstrap required, opened a new log
I20260812 03:03:37.170485 13544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:37.162037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59968 (local address 127.5.96.190:35095)
0812 03:03:37.162730 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:37.162747 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:37.162789 (+ 42us) server_negotiation.cc:407] Connection header received
0812 03:03:37.163562 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:37.163587 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:37.164089 (+ 502us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:37.164452 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:37.164834 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:37.166167 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:37.167059 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:37.167757 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:37.167913 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:37.168572 (+ 659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:37.168579 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:37.168595 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:03:37.169054 (+ 459us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:37.169121 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:03:37.169148 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:37.169262 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:37.169335 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 03:03:37.169605 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":125,"threads_started":1}
I20260812 03:03:37.170380 13541 raft_consensus.cc:348] T 00000000000000000000000000000000 P d27836aac8654660ac69f6ca1a1a6ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d27836aac8654660ac69f6ca1a1a6ffd" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35095 } }
I20260812 03:03:37.170611 13541 raft_consensus.cc:374] T 00000000000000000000000000000000 P d27836aac8654660ac69f6ca1a1a6ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:37.170678 13541 raft_consensus.cc:729] T 00000000000000000000000000000000 P d27836aac8654660ac69f6ca1a1a6ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d27836aac8654660ac69f6ca1a1a6ffd, State: Initialized, Role: FOLLOWER
I20260812 03:03:37.171084 13541 consensus_queue.cc:260] T 00000000000000000000000000000000 P d27836aac8654660ac69f6ca1a1a6ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d27836aac8654660ac69f6ca1a1a6ffd" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35095 } }
I20260812 03:03:37.171195 13541 raft_consensus.cc:388] T 00000000000000000000000000000000 P d27836aac8654660ac69f6ca1a1a6ffd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:03:37.171234 13541 raft_consensus.cc:482] T 00000000000000000000000000000000 P d27836aac8654660ac69f6ca1a1a6ffd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:03:37.171300 13541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d27836aac8654660ac69f6ca1a1a6ffd [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:37.172279 13541 raft_consensus.cc:504] T 00000000000000000000000000000000 P d27836aac8654660ac69f6ca1a1a6ffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d27836aac8654660ac69f6ca1a1a6ffd" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35095 } }
I20260812 03:03:37.172588 13541 leader_election.cc:302] T 00000000000000000000000000000000 P d27836aac8654660ac69f6ca1a1a6ffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d27836aac8654660ac69f6ca1a1a6ffd; no voters:
I20260812 03:03:37.172901 13541 leader_election.cc:288] T 00000000000000000000000000000000 P d27836aac8654660ac69f6ca1a1a6ffd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:03:37.172974 13548 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d27836aac8654660ac69f6ca1a1a6ffd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:37.173139 13548 raft_consensus.cc:686] T 00000000000000000000000000000000 P d27836aac8654660ac69f6ca1a1a6ffd [term 1 LEADER]: Becoming Leader. State: Replica: d27836aac8654660ac69f6ca1a1a6ffd, State: Running, Role: LEADER
I20260812 03:03:37.173173 5506 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:03:37.173425 13548 consensus_queue.cc:237] T 00000000000000000000000000000000 P d27836aac8654660ac69f6ca1a1a6ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d27836aac8654660ac69f6ca1a1a6ffd" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35095 } }
I20260812 03:03:37.174072 13541 sys_catalog.cc:564] T 00000000000000000000000000000000 P d27836aac8654660ac69f6ca1a1a6ffd [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:03:37.175104 13551 sys_catalog.cc:455] T 00000000000000000000000000000000 P d27836aac8654660ac69f6ca1a1a6ffd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d27836aac8654660ac69f6ca1a1a6ffd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d27836aac8654660ac69f6ca1a1a6ffd" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35095 } } }
I20260812 03:03:37.175225 13551 sys_catalog.cc:458] T 00000000000000000000000000000000 P d27836aac8654660ac69f6ca1a1a6ffd [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:37.175496 13550 sys_catalog.cc:455] T 00000000000000000000000000000000 P d27836aac8654660ac69f6ca1a1a6ffd [sys.catalog]: SysCatalogTable state changed. Reason: New leader d27836aac8654660ac69f6ca1a1a6ffd. Latest consensus state: current_term: 1 leader_uuid: "d27836aac8654660ac69f6ca1a1a6ffd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d27836aac8654660ac69f6ca1a1a6ffd" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35095 } } }
I20260812 03:03:37.175583 13561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:03:37.175598 13550 sys_catalog.cc:458] T 00000000000000000000000000000000 P d27836aac8654660ac69f6ca1a1a6ffd [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:37.178952 13561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:03:37.183796 13561 catalog_manager.cc:1347] Generated new cluster ID: 8bf6a7d5069a4b73bfe306bf6e3dba0b
I20260812 03:03:37.183854 13561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:03:37.194409 13561 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:03:37.195142 13561 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:03:37.201071 13561 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d27836aac8654660ac69f6ca1a1a6ffd: Generated new TSK 0
I20260812 03:03:37.201596 13561 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 03:03:37.227236 13584 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 03:03:37.227361 13584 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 03:03:37.227815 13544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:37.211553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59970 (local address 127.5.96.190:35095)
0812 03:03:37.211695 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:37.211699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:37.211896 (+ 197us) server_negotiation.cc:407] Connection header received
0812 03:03:37.212082 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:37.212084 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:37.212125 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:37.212198 (+ 73us) server_negotiation.cc:226] Negotiated authn=JWT
0812 03:03:37.212460 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:37.212989 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:37.214162 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:37.214368 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:37.214431 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:37.214742 (+ 311us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 03:03:37.227519 (+ 12777us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40235/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 03:03:37.227726 (+ 207us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59970: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40235/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":48}
W20260812 03:03:37.227878 13544 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59970: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40235/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 03:03:37.229274 13583 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:37.211476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:35095 (local address 127.0.0.1:59970)
0812 03:03:37.211818 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:37.211833 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:37.211955 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:37.212240 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:37.212243 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:37.212252 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0812 03:03:37.212368 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:37.212374 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:37.213083 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:37.213086 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:37.214065 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:37.214075 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:37.214458 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:37.214462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:37.214552 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:37.214600 (+ 48us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 03:03:37.227712 (+ 13112us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40235/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 03:03:37.227909 (+ 197us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.96.190:35095: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40235/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":82,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.5.96.190:35095: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40235/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 03:03:37.229964 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 13428
2026-08-12T03:03:37Z chronyd exiting
I20260812 03:03:37.270313 5506 test_util.cc:175] -----------------------------------------------
I20260812 03:03:37.270376 5506 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786503790637001-5506-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (297 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 03:03:37.271407 5506 mini_oidc.cc:152] Starting JWKS server
I20260812 03:03:37.272214 5506 webserver.cc:460] Webserver started at https://127.0.0.1:46519/ using document root <none> and password file <none>
I20260812 03:03:37.272296 5506 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 03:03:37.272496 5506 webserver.cc:460] Webserver started at http://127.0.0.1:44131/ using document root <none> and password file <none>
2026-08-12T03:03:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:03:37Z Disabled control of system clock
I20260812 03:03:37.281497 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:34605
--webserver_interface=127.5.96.190
--webserver_port=0
--builtin_ntp_servers=127.5.96.148:39695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:34605
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46519/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786503790637001-5506-0/testchainca.cert with env {}
W20260812 03:03:37.387501 13593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:37.387750 13593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:37.387787 13593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:37.391580 13593 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:03:37.391623 13593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:37.391636 13593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:37.391654 13593 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:03:37.391690 13593 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 03:03:37.391718 13593 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 03:03:37.391745 13593 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46519/jwks.json
I20260812 03:03:37.396775 13593 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:39695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:34605
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:34605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46519/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786503790637001-5506-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:37.397643 13593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:37.398514 13593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:37.406250 13602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:37.406267 13601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:37.406245 13605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:37.406574 13593 server_base.cc:1034] running on GCE node
I20260812 03:03:37.407123 13593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:37.408387 13593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:37.409564 13593 hybrid_clock.cc:648] HybridClock initialized: now 1786503817409493 us; error 92 us; skew 500 ppm
I20260812 03:03:37.411499 13593 webserver.cc:460] Webserver started at http://127.5.96.190:39955/ using document root <none> and password file <none>
I20260812 03:03:37.412091 13593 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:37.412160 13593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:37.412354 13593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:37.414346 13593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "e96130cab7cc4dc6ad9c376ee05d01e7"
format_stamp: "Formatted at 2026-08-12 03:03:37 on dist-test-slave-6bbx"
server_key: "211eab0055ed4aa2b8081acd86068ba4"
server_key_iv: "9c22bfeb13d2a852fb26216aad45c34b"
server_key_version: "encryptionkey@0"
I20260812 03:03:37.414878 13593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "e96130cab7cc4dc6ad9c376ee05d01e7"
format_stamp: "Formatted at 2026-08-12 03:03:37 on dist-test-slave-6bbx"
server_key: "211eab0055ed4aa2b8081acd86068ba4"
server_key_iv: "9c22bfeb13d2a852fb26216aad45c34b"
server_key_version: "encryptionkey@0"
I20260812 03:03:37.418529 13593 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 03:03:37.420733 13619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:37.421862 13593 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:03:37.421972 13593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "e96130cab7cc4dc6ad9c376ee05d01e7"
format_stamp: "Formatted at 2026-08-12 03:03:37 on dist-test-slave-6bbx"
server_key: "211eab0055ed4aa2b8081acd86068ba4"
server_key_iv: "9c22bfeb13d2a852fb26216aad45c34b"
server_key_version: "encryptionkey@0"
I20260812 03:03:37.422060 13593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:37.440521 13593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:37.441138 13593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:37.441299 13593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:37.447734 13593 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:34605
I20260812 03:03:37.447754 13697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:34605 every 8 connection(s)
I20260812 03:03:37.448585 13593 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:03:37.451512 13699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:37.456329 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 13593
I20260812 03:03:37.456423 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:03:37.456668 5506 external_mini_cluster.cc:1428] Setting key 0b34812a7fc76088922230e7ac2ca18e
I20260812 03:03:37.456794 13699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e96130cab7cc4dc6ad9c376ee05d01e7: Bootstrap starting.
I20260812 03:03:37.459376 13699 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e96130cab7cc4dc6ad9c376ee05d01e7: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:37.460273 13699 log.cc:824] T 00000000000000000000000000000000 P e96130cab7cc4dc6ad9c376ee05d01e7: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:37.462304 13699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e96130cab7cc4dc6ad9c376ee05d01e7: No bootstrap required, opened a new log
I20260812 03:03:37.466272 13702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:37.457854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40996 (local address 127.5.96.190:34605)
0812 03:03:37.458485 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:37.458503 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:37.458545 (+ 42us) server_negotiation.cc:407] Connection header received
0812 03:03:37.459316 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:37.459342 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:37.459869 (+ 527us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:37.460217 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:37.460653 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:37.461938 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:37.462811 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:37.463457 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:37.463623 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:37.464314 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:37.464321 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:37.464337 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:03:37.464808 (+ 471us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:37.464873 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:03:37.464899 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:37.464995 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:37.465083 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 03:03:37.465356 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":104,"threads_started":1}
I20260812 03:03:37.466413 13699 raft_consensus.cc:348] T 00000000000000000000000000000000 P e96130cab7cc4dc6ad9c376ee05d01e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96130cab7cc4dc6ad9c376ee05d01e7" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 34605 } }
I20260812 03:03:37.466647 13699 raft_consensus.cc:374] T 00000000000000000000000000000000 P e96130cab7cc4dc6ad9c376ee05d01e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:37.466717 13699 raft_consensus.cc:729] T 00000000000000000000000000000000 P e96130cab7cc4dc6ad9c376ee05d01e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e96130cab7cc4dc6ad9c376ee05d01e7, State: Initialized, Role: FOLLOWER
I20260812 03:03:37.467128 13699 consensus_queue.cc:260] T 00000000000000000000000000000000 P e96130cab7cc4dc6ad9c376ee05d01e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96130cab7cc4dc6ad9c376ee05d01e7" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 34605 } }
I20260812 03:03:37.467265 13699 raft_consensus.cc:388] T 00000000000000000000000000000000 P e96130cab7cc4dc6ad9c376ee05d01e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:03:37.467329 13699 raft_consensus.cc:482] T 00000000000000000000000000000000 P e96130cab7cc4dc6ad9c376ee05d01e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:03:37.467424 13699 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e96130cab7cc4dc6ad9c376ee05d01e7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:37.468343 13699 raft_consensus.cc:504] T 00000000000000000000000000000000 P e96130cab7cc4dc6ad9c376ee05d01e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96130cab7cc4dc6ad9c376ee05d01e7" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 34605 } }
I20260812 03:03:37.468659 13699 leader_election.cc:302] T 00000000000000000000000000000000 P e96130cab7cc4dc6ad9c376ee05d01e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e96130cab7cc4dc6ad9c376ee05d01e7; no voters:
I20260812 03:03:37.468784 5506 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:03:37.468966 13699 leader_election.cc:288] T 00000000000000000000000000000000 P e96130cab7cc4dc6ad9c376ee05d01e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:03:37.469038 13704 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e96130cab7cc4dc6ad9c376ee05d01e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:37.469230 13704 raft_consensus.cc:686] T 00000000000000000000000000000000 P e96130cab7cc4dc6ad9c376ee05d01e7 [term 1 LEADER]: Becoming Leader. State: Replica: e96130cab7cc4dc6ad9c376ee05d01e7, State: Running, Role: LEADER
I20260812 03:03:37.469511 13704 consensus_queue.cc:237] T 00000000000000000000000000000000 P e96130cab7cc4dc6ad9c376ee05d01e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96130cab7cc4dc6ad9c376ee05d01e7" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 34605 } }
I20260812 03:03:37.470168 13699 sys_catalog.cc:564] T 00000000000000000000000000000000 P e96130cab7cc4dc6ad9c376ee05d01e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:03:37.471313 13705 sys_catalog.cc:455] T 00000000000000000000000000000000 P e96130cab7cc4dc6ad9c376ee05d01e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e96130cab7cc4dc6ad9c376ee05d01e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96130cab7cc4dc6ad9c376ee05d01e7" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 34605 } } }
I20260812 03:03:37.471313 13706 sys_catalog.cc:455] T 00000000000000000000000000000000 P e96130cab7cc4dc6ad9c376ee05d01e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e96130cab7cc4dc6ad9c376ee05d01e7. Latest consensus state: current_term: 1 leader_uuid: "e96130cab7cc4dc6ad9c376ee05d01e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e96130cab7cc4dc6ad9c376ee05d01e7" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 34605 } } }
I20260812 03:03:37.471449 13705 sys_catalog.cc:458] T 00000000000000000000000000000000 P e96130cab7cc4dc6ad9c376ee05d01e7 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:37.471460 13706 sys_catalog.cc:458] T 00000000000000000000000000000000 P e96130cab7cc4dc6ad9c376ee05d01e7 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:37.471872 13722 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:03:37.474799 13722 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:03:37.480168 13722 catalog_manager.cc:1347] Generated new cluster ID: cc18ed97bf4647cdaaa6b29f307beb17
I20260812 03:03:37.480228 13722 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:03:37.499682 13722 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:03:37.500540 13722 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:03:37.511119 13722 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e96130cab7cc4dc6ad9c376ee05d01e7: Generated new TSK 0
I20260812 03:03:37.511842 13722 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 03:03:37.543730 13747 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 03:03:37.543792 13747 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 03:03:37.544431 13702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:37.526745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41008 (local address 127.5.96.190:34605)
0812 03:03:37.526862 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:37.526866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:37.527011 (+ 145us) server_negotiation.cc:407] Connection header received
0812 03:03:37.527141 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:37.527144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:37.527197 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:37.527301 (+ 104us) server_negotiation.cc:226] Negotiated authn=JWT
0812 03:03:37.527473 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:37.528313 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:37.529114 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:37.529452 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:37.529553 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:37.529787 (+ 234us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 03:03:37.544082 (+ 14295us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 03:03:37.544305 (+ 223us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41008: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":44}
W20260812 03:03:37.544481 13746 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:37.526680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:34605 (local address 127.0.0.1:41008)
0812 03:03:37.526936 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:37.526948 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:37.527046 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:37.527293 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:37.527294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:37.527300 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0812 03:03:37.527389 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:37.527393 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:37.528406 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:37.528408 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:37.529018 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:37.529025 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:37.529549 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:37.529551 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:37.529609 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:37.529629 (+ 20us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 03:03:37.544233 (+ 14604us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 03:03:37.544353 (+ 120us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.96.190:34605: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":103,"threads_started":1}
W20260812 03:03:37.544492 13702 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41008: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260812 03:03:37.545111 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 13593
2026-08-12T03:03:37Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (319 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 03:03:37.590515 5506 mini_oidc.cc:152] Starting JWKS server
I20260812 03:03:37.591336 5506 webserver.cc:460] Webserver started at https://127.0.0.1:33783/ using document root <none> and password file <none>
I20260812 03:03:37.591423 5506 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 03:03:37.591696 5506 webserver.cc:460] Webserver started at http://127.0.0.1:44379/ using document root <none> and password file <none>
2026-08-12T03:03:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:03:37Z Disabled control of system clock
I20260812 03:03:37.600944 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:42255
--webserver_interface=127.5.96.190
--webserver_port=0
--builtin_ntp_servers=127.5.96.148:46735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:42255
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33783/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 03:03:37.707537 13757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:37.707801 13757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:37.707841 13757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:37.711630 13757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:03:37.711686 13757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:37.711701 13757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:37.711715 13757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:03:37.711728 13757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 03:03:37.711747 13757 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 03:03:37.711763 13757 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33783/jwks.json
I20260812 03:03:37.716544 13757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:46735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:42255
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:42255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33783/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:37.717406 13757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:37.718617 13757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:37.726513 13763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:37.726512 13769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:37.726547 13765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:37.726594 13757 server_base.cc:1034] running on GCE node
I20260812 03:03:37.727195 13757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:37.728196 13757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:37.729348 13757 hybrid_clock.cc:648] HybridClock initialized: now 1786503817729328 us; error 32 us; skew 500 ppm
I20260812 03:03:37.730966 13757 webserver.cc:460] Webserver started at http://127.5.96.190:41091/ using document root <none> and password file <none>
I20260812 03:03:37.731460 13757 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:37.731520 13757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:37.731743 13757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:37.733609 13757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "02f08705a63248658af6262f0c6aa531"
format_stamp: "Formatted at 2026-08-12 03:03:37 on dist-test-slave-6bbx"
server_key: "b4e2a402bdc74cf6ffe65fbaf976264d"
server_key_iv: "8437478524dd06114dacd9bdac7156b9"
server_key_version: "encryptionkey@0"
I20260812 03:03:37.734107 13757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "02f08705a63248658af6262f0c6aa531"
format_stamp: "Formatted at 2026-08-12 03:03:37 on dist-test-slave-6bbx"
server_key: "b4e2a402bdc74cf6ffe65fbaf976264d"
server_key_iv: "8437478524dd06114dacd9bdac7156b9"
server_key_version: "encryptionkey@0"
I20260812 03:03:37.737417 13757 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 03:03:37.739524 13777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:37.740634 13757 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:37.740744 13757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "02f08705a63248658af6262f0c6aa531"
format_stamp: "Formatted at 2026-08-12 03:03:37 on dist-test-slave-6bbx"
server_key: "b4e2a402bdc74cf6ffe65fbaf976264d"
server_key_iv: "8437478524dd06114dacd9bdac7156b9"
server_key_version: "encryptionkey@0"
I20260812 03:03:37.740833 13757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:37.755924 13757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:37.756467 13757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:37.756615 13757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:37.762887 13757 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:42255
I20260812 03:03:37.762907 13869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:42255 every 8 connection(s)
I20260812 03:03:37.763706 13757 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:03:37.765683 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 13757
I20260812 03:03:37.765805 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:03:37.766072 5506 external_mini_cluster.cc:1428] Setting key 9ec88e2897ed66dcd5cc7590d35c0c67
I20260812 03:03:37.766852 13870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:37.772187 13870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02f08705a63248658af6262f0c6aa531: Bootstrap starting.
I20260812 03:03:37.774206 13870 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02f08705a63248658af6262f0c6aa531: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:37.774273 13874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:37.767321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42548 (local address 127.5.96.190:42255)
0812 03:03:37.767942 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:37.767957 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:37.767990 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:03:37.768593 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:37.768608 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:37.768964 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:37.769245 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:37.769630 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:37.770547 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:37.771388 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:37.771883 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:37.772024 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:37.772775 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:37.772779 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:37.772791 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:03:37.773124 (+ 333us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:37.773176 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:03:37.773194 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:37.773277 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:37.773418 (+ 141us) server_negotiation.cc:299] Negotiation successful
0812 03:03:37.773644 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":134,"threads_started":1}
I20260812 03:03:37.775055 13870 log.cc:824] T 00000000000000000000000000000000 P 02f08705a63248658af6262f0c6aa531: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:37.776911 5506 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:03:37.776964 13870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02f08705a63248658af6262f0c6aa531: No bootstrap required, opened a new log
I20260812 03:03:37.780272 13870 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02f08705a63248658af6262f0c6aa531 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f08705a63248658af6262f0c6aa531" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42255 } }
I20260812 03:03:37.780464 13870 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02f08705a63248658af6262f0c6aa531 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:37.780514 13870 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02f08705a63248658af6262f0c6aa531 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02f08705a63248658af6262f0c6aa531, State: Initialized, Role: FOLLOWER
I20260812 03:03:37.780889 13870 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02f08705a63248658af6262f0c6aa531 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f08705a63248658af6262f0c6aa531" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42255 } }
I20260812 03:03:37.781008 13870 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02f08705a63248658af6262f0c6aa531 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:03:37.781069 13870 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02f08705a63248658af6262f0c6aa531 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:03:37.781154 13870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02f08705a63248658af6262f0c6aa531 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:37.781922 13870 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02f08705a63248658af6262f0c6aa531 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f08705a63248658af6262f0c6aa531" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42255 } }
I20260812 03:03:37.782229 13870 leader_election.cc:302] T 00000000000000000000000000000000 P 02f08705a63248658af6262f0c6aa531 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02f08705a63248658af6262f0c6aa531; no voters:
I20260812 03:03:37.782477 13870 leader_election.cc:288] T 00000000000000000000000000000000 P 02f08705a63248658af6262f0c6aa531 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:03:37.782589 13877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02f08705a63248658af6262f0c6aa531 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:37.782817 13877 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02f08705a63248658af6262f0c6aa531 [term 1 LEADER]: Becoming Leader. State: Replica: 02f08705a63248658af6262f0c6aa531, State: Running, Role: LEADER
I20260812 03:03:37.783082 13877 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02f08705a63248658af6262f0c6aa531 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f08705a63248658af6262f0c6aa531" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42255 } }
I20260812 03:03:37.783514 13870 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02f08705a63248658af6262f0c6aa531 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:03:37.784523 13878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02f08705a63248658af6262f0c6aa531 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02f08705a63248658af6262f0c6aa531" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f08705a63248658af6262f0c6aa531" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42255 } } }
I20260812 03:03:37.784647 13878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02f08705a63248658af6262f0c6aa531 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:37.784919 13879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02f08705a63248658af6262f0c6aa531 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02f08705a63248658af6262f0c6aa531. Latest consensus state: current_term: 1 leader_uuid: "02f08705a63248658af6262f0c6aa531" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f08705a63248658af6262f0c6aa531" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42255 } } }
I20260812 03:03:37.785008 13879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02f08705a63248658af6262f0c6aa531 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:37.785022 13889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:03:37.787513 13889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:03:37.792609 13889 catalog_manager.cc:1347] Generated new cluster ID: a01e79ef106743769be57810de2f5080
I20260812 03:03:37.792656 13889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:03:37.811053 13889 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:03:37.811913 13889 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:03:37.815850 13889 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02f08705a63248658af6262f0c6aa531: Generated new TSK 0
I20260812 03:03:37.816427 13889 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:03:37.827389 13874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:37.822860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42560 (local address 127.5.96.190:42255)
0812 03:03:37.822970 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:37.822975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:37.823123 (+ 148us) server_negotiation.cc:407] Connection header received
0812 03:03:37.823284 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:37.823287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:37.823331 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:37.823415 (+ 84us) server_negotiation.cc:226] Negotiated authn=JWT
0812 03:03:37.823650 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:37.824185 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:37.825026 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:37.825269 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:37.825355 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:37.825654 (+ 299us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 03:03:37.827003 (+ 1349us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33783/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 03:03:37.827238 (+ 235us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42560: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33783/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":36}
W20260812 03:03:37.827446 13874 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42560: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33783/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 03:03:37.829406 13919 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:37.822788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:42255 (local address 127.0.0.1:42560)
0812 03:03:37.823047 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:37.823058 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:37.823160 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:37.823426 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:37.823430 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:37.823437 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0812 03:03:37.823553 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:37.823558 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:37.824298 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:37.824301 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:37.824919 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:37.824926 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:37.825389 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:37.825393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:37.825480 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:37.825502 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 03:03:37.827223 (+ 1721us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33783/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 03:03:37.829231 (+ 2008us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.96.190:42255: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33783/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":1804,"thread_start_us":99,"threads_started":1}
I20260812 03:03:37.829734 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 13757
E20260812 03:03:37.846414 13925 webserver.cc:533] Webserver: sslize failed:
E20260812 03:03:37.846478 13925 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-12T03:03:37Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (288 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T03:03:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:03:37Z Disabled control of system clock
I20260812 03:03:37.888540 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:37891
--webserver_interface=127.5.96.190
--webserver_port=0
--builtin_ntp_servers=127.5.96.148:37611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:37891
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786503790637001-5506-0/testchainca.cert with env {}
W20260812 03:03:37.994987 13932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:37.995225 13932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:37.995280 13932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:37.999126 13932 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:03:37.999174 13932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:37.999202 13932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:37.999225 13932 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:03:37.999253 13932 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 03:03:37.999276 13932 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 03:03:37.999301 13932 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 03:03:38.004406 13932 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:37611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:37891
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:37891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786503790637001-5506-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:38.005344 13932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:38.006289 13932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:38.014092 13943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:38.014056 13944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:38.014070 13947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:38.014365 13932 server_base.cc:1034] running on GCE node
I20260812 03:03:38.014891 13932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:38.015916 13932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:38.017067 13932 hybrid_clock.cc:648] HybridClock initialized: now 1786503818017054 us; error 24 us; skew 500 ppm
I20260812 03:03:38.018718 13932 webserver.cc:460] Webserver started at http://127.5.96.190:38247/ using document root <none> and password file <none>
I20260812 03:03:38.019229 13932 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:38.019290 13932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:38.019474 13932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:38.021360 13932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "e824df29061f4088a370c7389b30738f"
format_stamp: "Formatted at 2026-08-12 03:03:38 on dist-test-slave-6bbx"
server_key: "0ef900b841f490445b66114915cc25c5"
server_key_iv: "b9fc07c51c17b2d1dac4cf9f4a8e70f1"
server_key_version: "encryptionkey@0"
I20260812 03:03:38.021875 13932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "e824df29061f4088a370c7389b30738f"
format_stamp: "Formatted at 2026-08-12 03:03:38 on dist-test-slave-6bbx"
server_key: "0ef900b841f490445b66114915cc25c5"
server_key_iv: "b9fc07c51c17b2d1dac4cf9f4a8e70f1"
server_key_version: "encryptionkey@0"
I20260812 03:03:38.025247 13932 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260812 03:03:38.027370 13957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:38.028484 13932 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:38.028592 13932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "e824df29061f4088a370c7389b30738f"
format_stamp: "Formatted at 2026-08-12 03:03:38 on dist-test-slave-6bbx"
server_key: "0ef900b841f490445b66114915cc25c5"
server_key_iv: "b9fc07c51c17b2d1dac4cf9f4a8e70f1"
server_key_version: "encryptionkey@0"
I20260812 03:03:38.028680 13932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:38.051357 13932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:38.051942 13932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:38.052093 13932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:38.058372 13932 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:37891
I20260812 03:03:38.058385 14038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:37891 every 8 connection(s)
I20260812 03:03:38.059194 13932 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:03:38.062275 14039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:38.063315 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 13932
I20260812 03:03:38.063431 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:03:38.063730 5506 external_mini_cluster.cc:1428] Setting key 24d32a926bdeba6e714c3b633fe60fef
I20260812 03:03:38.068503 14039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e824df29061f4088a370c7389b30738f: Bootstrap starting.
I20260812 03:03:38.070945 14039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e824df29061f4088a370c7389b30738f: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:38.071738 14039 log.cc:824] T 00000000000000000000000000000000 P e824df29061f4088a370c7389b30738f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:38.073197 14044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:38.064955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45952 (local address 127.5.96.190:37891)
0812 03:03:38.065597 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:38.065621 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:38.065663 (+ 42us) server_negotiation.cc:407] Connection header received
0812 03:03:38.066442 (+ 779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:38.066466 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:38.066951 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:38.067302 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:38.067734 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:38.069142 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:38.069967 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:38.070613 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:38.070768 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:38.071452 (+ 684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:38.071458 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:38.071473 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:03:38.071973 (+ 500us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:38.072043 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:03:38.072071 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:38.072158 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:38.072300 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 03:03:38.072538 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":106,"threads_started":1}
I20260812 03:03:38.073586 14039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e824df29061f4088a370c7389b30738f: No bootstrap required, opened a new log
I20260812 03:03:38.075680 5506 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:03:38.077222 14039 raft_consensus.cc:348] T 00000000000000000000000000000000 P e824df29061f4088a370c7389b30738f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e824df29061f4088a370c7389b30738f" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37891 } }
I20260812 03:03:38.077414 14039 raft_consensus.cc:374] T 00000000000000000000000000000000 P e824df29061f4088a370c7389b30738f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:38.077452 14039 raft_consensus.cc:729] T 00000000000000000000000000000000 P e824df29061f4088a370c7389b30738f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e824df29061f4088a370c7389b30738f, State: Initialized, Role: FOLLOWER
I20260812 03:03:38.077786 14039 consensus_queue.cc:260] T 00000000000000000000000000000000 P e824df29061f4088a370c7389b30738f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e824df29061f4088a370c7389b30738f" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37891 } }
I20260812 03:03:38.077893 14039 raft_consensus.cc:388] T 00000000000000000000000000000000 P e824df29061f4088a370c7389b30738f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:03:38.077934 14039 raft_consensus.cc:482] T 00000000000000000000000000000000 P e824df29061f4088a370c7389b30738f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:03:38.077998 14039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e824df29061f4088a370c7389b30738f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:38.078714 14039 raft_consensus.cc:504] T 00000000000000000000000000000000 P e824df29061f4088a370c7389b30738f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e824df29061f4088a370c7389b30738f" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37891 } }
I20260812 03:03:38.078996 14039 leader_election.cc:302] T 00000000000000000000000000000000 P e824df29061f4088a370c7389b30738f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e824df29061f4088a370c7389b30738f; no voters:
I20260812 03:03:38.079221 14039 leader_election.cc:288] T 00000000000000000000000000000000 P e824df29061f4088a370c7389b30738f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:03:38.079312 14049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e824df29061f4088a370c7389b30738f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:38.079558 14049 raft_consensus.cc:686] T 00000000000000000000000000000000 P e824df29061f4088a370c7389b30738f [term 1 LEADER]: Becoming Leader. State: Replica: e824df29061f4088a370c7389b30738f, State: Running, Role: LEADER
I20260812 03:03:38.079864 14049 consensus_queue.cc:237] T 00000000000000000000000000000000 P e824df29061f4088a370c7389b30738f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e824df29061f4088a370c7389b30738f" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37891 } }
I20260812 03:03:38.080252 14039 sys_catalog.cc:564] T 00000000000000000000000000000000 P e824df29061f4088a370c7389b30738f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:03:38.081277 14053 sys_catalog.cc:455] T 00000000000000000000000000000000 P e824df29061f4088a370c7389b30738f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e824df29061f4088a370c7389b30738f. Latest consensus state: current_term: 1 leader_uuid: "e824df29061f4088a370c7389b30738f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e824df29061f4088a370c7389b30738f" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37891 } } }
I20260812 03:03:38.081398 14053 sys_catalog.cc:458] T 00000000000000000000000000000000 P e824df29061f4088a370c7389b30738f [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:38.081763 14065 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:03:38.081677 14052 sys_catalog.cc:455] T 00000000000000000000000000000000 P e824df29061f4088a370c7389b30738f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e824df29061f4088a370c7389b30738f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e824df29061f4088a370c7389b30738f" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37891 } } }
I20260812 03:03:38.082983 14052 sys_catalog.cc:458] T 00000000000000000000000000000000 P e824df29061f4088a370c7389b30738f [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:38.084513 14065 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:03:38.089538 14065 catalog_manager.cc:1347] Generated new cluster ID: df8606873c4b44b4952d48bbf11d4ce5
I20260812 03:03:38.089596 14065 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:03:38.104669 14065 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:03:38.105747 14065 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:03:38.112470 14065 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e824df29061f4088a370c7389b30738f: Generated new TSK 0
I20260812 03:03:38.113124 14065 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:03:38.224952 14044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:38.128521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45954 (local address 127.5.96.190:37891)
0812 03:03:38.128638 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:38.128642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:38.128776 (+ 134us) server_negotiation.cc:407] Connection header received
0812 03:03:38.128902 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:38.128905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:38.128957 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:38.129059 (+ 102us) server_negotiation.cc:226] Negotiated authn=JWT
0812 03:03:38.129234 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:38.130057 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:38.130839 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:38.131183 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:38.131288 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:38.131523 (+ 235us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 03:03:38.224575 (+ 93052us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 03:03:38.224832 (+ 257us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45954: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":43}
W20260812 03:03:38.225024 14044 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45954: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 03:03:38.225400 14088 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:38.128439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37891 (local address 127.0.0.1:45954)
0812 03:03:38.128702 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:38.128714 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:38.128805 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:38.129051 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:38.129053 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:38.129058 (+ 5us) client_negotiation.cc:189] Negotiated authn=JWT
0812 03:03:38.129143 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:38.129147 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:38.130141 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:38.130143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:38.130747 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:38.130753 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:38.131294 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:38.131297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:38.131364 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:38.131385 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 03:03:38.224804 (+ 93419us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 03:03:38.224978 (+ 174us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.96.190:37891: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":95,"threads_started":1}
I20260812 03:03:38.226058 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 13932
2026-08-12T03:03:38Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (387 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (120 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (119 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:03:38 dist-test-slave-6bbx krb5kdc[14105](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:03:38 dist-test-slave-6bbx krb5kdc[14105](info): set up 2 sockets
Aug 12 03:03:38 dist-test-slave-6bbx krb5kdc[14105](info): commencing operation
krb5kdc: starting...
W20260812 03:03:39.630760 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:03:39 dist-test-slave-6bbx krb5kdc[14105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:03:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:03:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:03:39.738722 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:33567
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:36443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:33567
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:39.847060 14126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:39.847291 14126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:39.847345 14126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:39.851222 14126 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:03:39.851272 14126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:39.851298 14126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:39.851320 14126 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:03:39.851346 14126 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:03:39.856381 14126 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:36443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:33567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:33567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:39.857295 14126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:39.858244 14126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:39.866549 14135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:39.866611 14139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:39.866613 14126 server_base.cc:1034] running on GCE node
W20260812 03:03:39.866556 14137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:39.867352 14126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:39.868422 14126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:39.869590 14126 hybrid_clock.cc:648] HybridClock initialized: now 1786503819869575 us; error 25 us; skew 500 ppm
Aug 12 03:03:39 dist-test-slave-6bbx krb5kdc[14105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503819, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:39.872195 14126 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:03:39.873050 14126 webserver.cc:460] Webserver started at http://127.5.96.190:33943/ using document root <none> and password file <none>
I20260812 03:03:39.873529 14126 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:39.873584 14126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:39.873756 14126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:39.875737 14126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "e122fc37fa4444d5a677a414947bc48a"
format_stamp: "Formatted at 2026-08-12 03:03:39 on dist-test-slave-6bbx"
server_key: "dd6c8114b44ca213c74335a247bc2370"
server_key_iv: "b75ec428b192033b7133234a2b42e7e1"
server_key_version: "encryptionkey@0"
I20260812 03:03:39.876260 14126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "e122fc37fa4444d5a677a414947bc48a"
format_stamp: "Formatted at 2026-08-12 03:03:39 on dist-test-slave-6bbx"
server_key: "dd6c8114b44ca213c74335a247bc2370"
server_key_iv: "b75ec428b192033b7133234a2b42e7e1"
server_key_version: "encryptionkey@0"
I20260812 03:03:39.879566 14126 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20260812 03:03:39.881742 14151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:39.882864 14126 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:39.882977 14126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "e122fc37fa4444d5a677a414947bc48a"
format_stamp: "Formatted at 2026-08-12 03:03:39 on dist-test-slave-6bbx"
server_key: "dd6c8114b44ca213c74335a247bc2370"
server_key_iv: "b75ec428b192033b7133234a2b42e7e1"
server_key_version: "encryptionkey@0"
I20260812 03:03:39.883073 14126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:39.896651 14126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:39.899048 14126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:39.899207 14126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:39.905757 14126 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:33567
I20260812 03:03:39.905781 14238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:33567 every 8 connection(s)
I20260812 03:03:39.906579 14126 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:03:39.909147 14240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:39.913565 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 14126
I20260812 03:03:39.913688 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:03:39.914011 5506 external_mini_cluster.cc:1428] Setting key f746ab3e9e668839ed691f886d96095a
I20260812 03:03:39.914521 14240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e122fc37fa4444d5a677a414947bc48a: Bootstrap starting.
I20260812 03:03:39.916736 14240 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e122fc37fa4444d5a677a414947bc48a: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:39.917618 14240 log.cc:824] T 00000000000000000000000000000000 P e122fc37fa4444d5a677a414947bc48a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:39.919481 14240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e122fc37fa4444d5a677a414947bc48a: No bootstrap required, opened a new log
Aug 12 03:03:39 dist-test-slave-6bbx krb5kdc[14105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:39.923349 14240 raft_consensus.cc:348] T 00000000000000000000000000000000 P e122fc37fa4444d5a677a414947bc48a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e122fc37fa4444d5a677a414947bc48a" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33567 } }
I20260812 03:03:39.923545 14240 raft_consensus.cc:374] T 00000000000000000000000000000000 P e122fc37fa4444d5a677a414947bc48a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:39.923600 14240 raft_consensus.cc:729] T 00000000000000000000000000000000 P e122fc37fa4444d5a677a414947bc48a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e122fc37fa4444d5a677a414947bc48a, State: Initialized, Role: FOLLOWER
I20260812 03:03:39.924005 14240 consensus_queue.cc:260] T 00000000000000000000000000000000 P e122fc37fa4444d5a677a414947bc48a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e122fc37fa4444d5a677a414947bc48a" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33567 } }
I20260812 03:03:39.924139 14240 raft_consensus.cc:388] T 00000000000000000000000000000000 P e122fc37fa4444d5a677a414947bc48a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:03:39.924201 14240 raft_consensus.cc:482] T 00000000000000000000000000000000 P e122fc37fa4444d5a677a414947bc48a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:03:39.924312 14240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e122fc37fa4444d5a677a414947bc48a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:39.925037 14240 raft_consensus.cc:504] T 00000000000000000000000000000000 P e122fc37fa4444d5a677a414947bc48a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e122fc37fa4444d5a677a414947bc48a" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33567 } }
I20260812 03:03:39.925369 14240 leader_election.cc:302] T 00000000000000000000000000000000 P e122fc37fa4444d5a677a414947bc48a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e122fc37fa4444d5a677a414947bc48a; no voters:
I20260812 03:03:39.925688 14240 leader_election.cc:288] T 00000000000000000000000000000000 P e122fc37fa4444d5a677a414947bc48a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:03:39.925787 14251 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e122fc37fa4444d5a677a414947bc48a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:39.926015 14251 raft_consensus.cc:686] T 00000000000000000000000000000000 P e122fc37fa4444d5a677a414947bc48a [term 1 LEADER]: Becoming Leader. State: Replica: e122fc37fa4444d5a677a414947bc48a, State: Running, Role: LEADER
I20260812 03:03:39.926275 14251 consensus_queue.cc:237] T 00000000000000000000000000000000 P e122fc37fa4444d5a677a414947bc48a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e122fc37fa4444d5a677a414947bc48a" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33567 } }
I20260812 03:03:39.926879 14240 sys_catalog.cc:564] T 00000000000000000000000000000000 P e122fc37fa4444d5a677a414947bc48a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:03:39.927260 14251 sys_catalog.cc:455] T 00000000000000000000000000000000 P e122fc37fa4444d5a677a414947bc48a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e122fc37fa4444d5a677a414947bc48a. Latest consensus state: current_term: 1 leader_uuid: "e122fc37fa4444d5a677a414947bc48a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e122fc37fa4444d5a677a414947bc48a" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33567 } } }
I20260812 03:03:39.927471 14251 sys_catalog.cc:458] T 00000000000000000000000000000000 P e122fc37fa4444d5a677a414947bc48a [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:39.927644 14253 sys_catalog.cc:455] T 00000000000000000000000000000000 P e122fc37fa4444d5a677a414947bc48a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e122fc37fa4444d5a677a414947bc48a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e122fc37fa4444d5a677a414947bc48a" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33567 } } }
I20260812 03:03:39.927757 14253 sys_catalog.cc:458] T 00000000000000000000000000000000 P e122fc37fa4444d5a677a414947bc48a [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:39.928114 14244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:39.915234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40384 (local address 127.5.96.190:33567)
0812 03:03:39.915606 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:39.915621 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:39.915654 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:03:39.916286 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:39.916302 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:39.916596 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:39.916888 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:39.917691 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:39.918620 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:39.919760 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:39.920204 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:39.920315 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:39.922872 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:39.922877 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:39.922888 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:39.922914 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:39.924552 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:39.924952 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:39.924958 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:39.924963 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:39.925071 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:39.925321 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:39.925324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:39.925326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:39.925758 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:39.925947 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:39.926563 (+ 616us) server_negotiation.cc:299] Negotiation successful
0812 03:03:39.926851 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":97,"threads_started":1}
W20260812 03:03:39.931516 14275 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e122fc37fa4444d5a677a414947bc48a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:03:39.931594 14275 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:03:39.931694 14260 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:03:39.932379 14260 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:03:39.937099 14260 catalog_manager.cc:1347] Generated new cluster ID: b4da34dab2514326a50a2c43528fdeca
I20260812 03:03:39.937147 14260 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:03:39.975916 14260 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:03:39.976807 14260 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:03:39.982851 14260 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e122fc37fa4444d5a677a414947bc48a: Generated new TSK 0
I20260812 03:03:39.983421 14260 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.96.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
I20260812 03:03:40.014462 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:0
--local_ip_for_outbound_sockets=127.5.96.129
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:33567
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:36443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:40.120772 14283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:40.120985 14283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:40.121027 14283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:40.124899 14283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:40.124944 14283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:40.125011 14283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:03:40.130129 14283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:36443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:33567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:40.131011 14283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:40.131970 14283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:40.140691 14297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:40.140715 14291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:40.140715 14292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:40.140857 14283 server_base.cc:1034] running on GCE node
I20260812 03:03:40.141383 14283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:40.141955 14283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:40.143098 14283 hybrid_clock.cc:648] HybridClock initialized: now 1786503820143082 us; error 27 us; skew 500 ppm
Aug 12 03:03:40 dist-test-slave-6bbx krb5kdc[14105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503820, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:40.148248 14283 init.cc:377] Logged in from keytab as kudu/127.5.96.129@KRBTEST.COM (short username kudu)
I20260812 03:03:40.149024 14283 webserver.cc:460] Webserver started at http://127.5.96.129:44365/ using document root <none> and password file <none>
I20260812 03:03:40.149520 14283 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:40.149581 14283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:40.149753 14283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:40.151559 14283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-0/data/instance:
uuid: "3011592a756649c1b3b2a3a3564b90ff"
format_stamp: "Formatted at 2026-08-12 03:03:40 on dist-test-slave-6bbx"
server_key: "a933a3364e668bbd0ecd2244e5ecb222"
server_key_iv: "fe0966591e4d8b2061d93e28e087c549"
server_key_version: "encryptionkey@0"
I20260812 03:03:40.152091 14283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance:
uuid: "3011592a756649c1b3b2a3a3564b90ff"
format_stamp: "Formatted at 2026-08-12 03:03:40 on dist-test-slave-6bbx"
server_key: "a933a3364e668bbd0ecd2244e5ecb222"
server_key_iv: "fe0966591e4d8b2061d93e28e087c549"
server_key_version: "encryptionkey@0"
I20260812 03:03:40.155365 14283 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:03:40.157560 14311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:40.158686 14283 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:40.158787 14283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "3011592a756649c1b3b2a3a3564b90ff"
format_stamp: "Formatted at 2026-08-12 03:03:40 on dist-test-slave-6bbx"
server_key: "a933a3364e668bbd0ecd2244e5ecb222"
server_key_iv: "fe0966591e4d8b2061d93e28e087c549"
server_key_version: "encryptionkey@0"
I20260812 03:03:40.158871 14283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:40.183358 14283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:40.185839 14283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:40.185992 14283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:40.186512 14283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:40.187278 14283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:40.187325 14283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:40.187376 14283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:40.187403 14283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:40.195470 14283 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:39595
I20260812 03:03:40.195494 14511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:39595 every 8 connection(s)
I20260812 03:03:40.196265 14283 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:03:40.199895 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 14283
I20260812 03:03:40.200008 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance
I20260812 03:03:40.200301 5506 external_mini_cluster.cc:1428] Setting key 8319891c644ca19724e7086ecfc69808
Aug 12 03:03:40 dist-test-slave-6bbx krb5kdc[14105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503820, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:40.209899 14244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:40.197869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:57743 (local address 127.5.96.190:33567)
0812 03:03:40.197988 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:40.197991 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:40.198742 (+ 751us) server_negotiation.cc:407] Connection header received
0812 03:03:40.200172 (+ 1430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:40.200176 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:40.200233 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:40.200310 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:40.201632 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:40.202199 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:40.203687 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:40.204028 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:40.204133 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:40.206558 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:40.206561 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:40.206563 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:40.206586 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:40.207962 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:40.208522 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:40.208524 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:40.208525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:40.208594 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:40.208930 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:40.208932 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:40.208933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:40.209086 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:40.209153 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:40.209692 (+ 539us) server_negotiation.cc:299] Negotiation successful
0812 03:03:40.209783 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
WARNING: no policy specified for kudu/127.5.96.130@KRBTEST.COM; defaulting to no policy
I20260812 03:03:40.210924 14516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:40.198197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33567 (local address 127.5.96.129:57743)
0812 03:03:40.198607 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:40.198646 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:40.199927 (+ 1281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:40.200513 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:40.200522 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:40.200966 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:40.201495 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:40.201514 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:40.202333 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:40.202337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:40.203545 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:40.203557 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:40.204160 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:40.204163 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:40.204300 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:40.204797 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:40.204813 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:40.206383 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:40.208070 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:40.208076 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:40.208090 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:40.208434 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:40.208669 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:40.208672 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:40.208674 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:40.208848 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:40.209185 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:40.209191 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:40.209475 (+ 284us) client_negotiation.cc:769] Sending connection context
0812 03:03:40.209676 (+ 201us) client_negotiation.cc:240] Negotiation successful
0812 03:03:40.209978 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":105,"threads_started":1}
I20260812 03:03:40.211978 14512 heartbeater.cc:347] Connected to a master server at 127.5.96.190:33567
I20260812 03:03:40.212185 14512 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:40.212680 14512 heartbeater.cc:511] Master 127.5.96.190:33567 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.130@KRBTEST.COM" created.
I20260812 03:03:40.214155 14182 ts_manager.cc:194] Registered new tserver with Master: 3011592a756649c1b3b2a3a3564b90ff (127.5.96.129:39595)
I20260812 03:03:40.215219 14182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.129@KRBTEST.COM'} at 127.5.96.129:57743
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
I20260812 03:03:40.225941 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:0
--local_ip_for_outbound_sockets=127.5.96.130
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:33567
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:36443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:40.330307 14523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:40.330554 14523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:40.330598 14523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:40.334448 14523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:40.334493 14523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:40.334546 14523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:03:40.339345 14523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:36443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:33567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:40.340207 14523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:40.341099 14523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:40.349685 14536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:40.349702 14531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:40.349732 14532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:40.349990 14523 server_base.cc:1034] running on GCE node
I20260812 03:03:40.350343 14523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:40.350907 14523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:40.352056 14523 hybrid_clock.cc:648] HybridClock initialized: now 1786503820352033 us; error 31 us; skew 500 ppm
Aug 12 03:03:40 dist-test-slave-6bbx krb5kdc[14105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503820, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:40.354485 14523 init.cc:377] Logged in from keytab as kudu/127.5.96.130@KRBTEST.COM (short username kudu)
I20260812 03:03:40.355238 14523 webserver.cc:460] Webserver started at http://127.5.96.130:37693/ using document root <none> and password file <none>
I20260812 03:03:40.355759 14523 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:40.355827 14523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:40.356020 14523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:40.357853 14523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-1/data/instance:
uuid: "fb26f5e1dc5a40f5b45e5a249d8e8dbf"
format_stamp: "Formatted at 2026-08-12 03:03:40 on dist-test-slave-6bbx"
server_key: "c075c932269fd02aacbdf0f2899296c1"
server_key_iv: "3fcc7e2fed9d1e1df8bc928a1578a5d1"
server_key_version: "encryptionkey@0"
I20260812 03:03:40.358363 14523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance:
uuid: "fb26f5e1dc5a40f5b45e5a249d8e8dbf"
format_stamp: "Formatted at 2026-08-12 03:03:40 on dist-test-slave-6bbx"
server_key: "c075c932269fd02aacbdf0f2899296c1"
server_key_iv: "3fcc7e2fed9d1e1df8bc928a1578a5d1"
server_key_version: "encryptionkey@0"
I20260812 03:03:40.361742 14523 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260812 03:03:40.364033 14545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:40.365121 14523 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:03:40.365226 14523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "fb26f5e1dc5a40f5b45e5a249d8e8dbf"
format_stamp: "Formatted at 2026-08-12 03:03:40 on dist-test-slave-6bbx"
server_key: "c075c932269fd02aacbdf0f2899296c1"
server_key_iv: "3fcc7e2fed9d1e1df8bc928a1578a5d1"
server_key_version: "encryptionkey@0"
I20260812 03:03:40.365309 14523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:40.374501 14523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:40.376868 14523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:40.377017 14523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:40.377508 14523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:40.378228 14523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:40.378278 14523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:40.378326 14523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:40.378352 14523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:40.386416 14523 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:45117
I20260812 03:03:40.386438 14746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:45117 every 8 connection(s)
I20260812 03:03:40.387153 14523 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:03:40.391381 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 14523
I20260812 03:03:40.391475 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance
I20260812 03:03:40.391765 5506 external_mini_cluster.cc:1428] Setting key ea5fe3180cb5fa008697dad8a3b8bceb
Aug 12 03:03:40 dist-test-slave-6bbx krb5kdc[14105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503820, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:40.399794 14244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:40.388756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:54043 (local address 127.5.96.190:33567)
0812 03:03:40.388889 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:40.388894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:40.389673 (+ 779us) server_negotiation.cc:407] Connection header received
0812 03:03:40.390880 (+ 1207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:40.390883 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:40.390928 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:40.390998 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:40.392122 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:40.392727 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:40.393888 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:40.394193 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:40.394293 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:40.396613 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:40.396616 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:40.396618 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:40.396640 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:40.397893 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:40.398386 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:40.398389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:40.398390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:40.398466 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:40.398781 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:40.398785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:40.398786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:40.398955 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:40.399039 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:40.399487 (+ 448us) server_negotiation.cc:299] Negotiation successful
0812 03:03:40.399607 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:03:40.400545 14756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:40.389117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33567 (local address 127.5.96.130:54043)
0812 03:03:40.389539 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:40.389578 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:40.390707 (+ 1129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:40.391131 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:40.391137 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:40.391467 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:40.391982 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:40.391997 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:40.392851 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:40.392855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:40.393763 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:40.393774 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:40.394312 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:40.394314 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:40.394443 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:40.394946 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:40.394966 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:40.396457 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:40.398000 (+ 1543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:40.398006 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:40.398020 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:40.398294 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:40.398547 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:40.398549 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:40.398551 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:40.398694 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:40.399054 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:40.399060 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:40.399275 (+ 215us) client_negotiation.cc:769] Sending connection context
0812 03:03:40.399428 (+ 153us) client_negotiation.cc:240] Negotiation successful
0812 03:03:40.399739 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":106,"threads_started":1}
WARNING: no policy specified for kudu/127.5.96.131@KRBTEST.COM; defaulting to no policy
I20260812 03:03:40.401501 14748 heartbeater.cc:347] Connected to a master server at 127.5.96.190:33567
I20260812 03:03:40.401712 14748 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:40.402140 14748 heartbeater.cc:511] Master 127.5.96.190:33567 requested a full tablet report, sending...
I20260812 03:03:40.403062 14182 ts_manager.cc:194] Registered new tserver with Master: fb26f5e1dc5a40f5b45e5a249d8e8dbf (127.5.96.130:45117)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.131@KRBTEST.COM" created.
I20260812 03:03:40.403730 14182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.130@KRBTEST.COM'} at 127.5.96.130:54043
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
I20260812 03:03:40.415541 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:0
--local_ip_for_outbound_sockets=127.5.96.131
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:33567
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:36443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:40.523813 14760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:40.524017 14760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:40.524055 14760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:40.527925 14760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:40.527968 14760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:40.528021 14760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:03:40.532958 14760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:36443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:33567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:40.533769 14760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:40.534597 14760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:40.543278 14768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:40.543284 14773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:40.543290 14769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:40.543534 14760 server_base.cc:1034] running on GCE node
I20260812 03:03:40.543998 14760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:40.544571 14760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:40.545723 14760 hybrid_clock.cc:648] HybridClock initialized: now 1786503820545709 us; error 27 us; skew 500 ppm
Aug 12 03:03:40 dist-test-slave-6bbx krb5kdc[14105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503820, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:40.548300 14760 init.cc:377] Logged in from keytab as kudu/127.5.96.131@KRBTEST.COM (short username kudu)
I20260812 03:03:40.549063 14760 webserver.cc:460] Webserver started at http://127.5.96.131:38151/ using document root <none> and password file <none>
I20260812 03:03:40.549558 14760 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:40.549620 14760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:40.549798 14760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:40.551601 14760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-2/data/instance:
uuid: "bc837d19fdce40109199e0fa83d1e4d5"
format_stamp: "Formatted at 2026-08-12 03:03:40 on dist-test-slave-6bbx"
server_key: "3ba45323caacb5290fca140e3bf462a7"
server_key_iv: "50c60fd10ac450ab45720d72d6d558c9"
server_key_version: "encryptionkey@0"
I20260812 03:03:40.552150 14760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance:
uuid: "bc837d19fdce40109199e0fa83d1e4d5"
format_stamp: "Formatted at 2026-08-12 03:03:40 on dist-test-slave-6bbx"
server_key: "3ba45323caacb5290fca140e3bf462a7"
server_key_iv: "50c60fd10ac450ab45720d72d6d558c9"
server_key_version: "encryptionkey@0"
I20260812 03:03:40.555406 14760 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:03:40.557555 14783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:40.558598 14760 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:40.558709 14760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "bc837d19fdce40109199e0fa83d1e4d5"
format_stamp: "Formatted at 2026-08-12 03:03:40 on dist-test-slave-6bbx"
server_key: "3ba45323caacb5290fca140e3bf462a7"
server_key_iv: "50c60fd10ac450ab45720d72d6d558c9"
server_key_version: "encryptionkey@0"
I20260812 03:03:40.558799 14760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:40.572871 14760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:40.575316 14760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:40.575470 14760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:40.576035 14760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:40.576776 14760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:40.576820 14760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:40.576869 14760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:40.576896 14760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:40.584964 14760 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:41537
I20260812 03:03:40.584993 14956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:41537 every 8 connection(s)
I20260812 03:03:40.585718 14760 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:03:40.590911 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 14760
I20260812 03:03:40.591010 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance
I20260812 03:03:40.591279 5506 external_mini_cluster.cc:1428] Setting key 118e7909e0869f0325e03e2411de488d
Aug 12 03:03:40 dist-test-slave-6bbx krb5kdc[14105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503820, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:40.597802 14244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:40.587362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:50969 (local address 127.5.96.190:33567)
0812 03:03:40.587493 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:40.587497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:40.588260 (+ 763us) server_negotiation.cc:407] Connection header received
0812 03:03:40.589532 (+ 1272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:40.589537 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:40.589594 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:40.589664 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:40.590727 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:40.591288 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:40.592588 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:40.592809 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:40.592897 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:40.595082 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:40.595085 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:40.595087 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:40.595110 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:40.596242 (+ 1132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:40.596664 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:40.596667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:40.596668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:40.596747 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:40.596989 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:40.596991 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:40.596992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:40.597142 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:40.597208 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:40.597583 (+ 375us) server_negotiation.cc:299] Negotiation successful
0812 03:03:40.597676 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:03:40.598456 14960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:40.587691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33567 (local address 127.5.96.131:50969)
0812 03:03:40.588123 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:40.588160 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:40.589333 (+ 1173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:40.589836 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:40.589843 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:40.590154 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:40.590582 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:40.590594 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:40.591421 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:40.591425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:40.592476 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:40.592486 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:40.592938 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:40.592942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:40.593130 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:40.593595 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:40.593609 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:40.594935 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:40.596347 (+ 1412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:40.596352 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:40.596362 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:40.596591 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:40.596809 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:40.596811 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:40.596812 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:40.596926 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:40.597207 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:40.597211 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:40.597412 (+ 201us) client_negotiation.cc:769] Sending connection context
0812 03:03:40.597550 (+ 138us) client_negotiation.cc:240] Negotiation successful
0812 03:03:40.597763 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":111,"threads_started":1}
I20260812 03:03:40.599448 14958 heartbeater.cc:347] Connected to a master server at 127.5.96.190:33567
I20260812 03:03:40.599632 14958 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:40.600050 14958 heartbeater.cc:511] Master 127.5.96.190:33567 requested a full tablet report, sending...
I20260812 03:03:40.600962 14182 ts_manager.cc:194] Registered new tserver with Master: bc837d19fdce40109199e0fa83d1e4d5 (127.5.96.131:41537)
I20260812 03:03:40.601523 14182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.131@KRBTEST.COM'} at 127.5.96.131:50969
I20260812 03:03:40.604554 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/test-admin.keytab.
Aug 12 03:03:40 dist-test-slave-6bbx krb5kdc[14105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:40.618667 5506 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 03:03:40.621284 14966 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:40.619849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43596 (local address 127.5.96.129:39595)
0812 03:03:40.620179 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:40.620187 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:40.620206 (+ 19us) server_negotiation.cc:407] Connection header received
0812 03:03:40.620250 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:40.620255 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:40.620400 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:40.620519 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:40.621071 (+ 552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43596: BlockingRecv error: recv got EOF from 127.0.0.1:43596 (error 108)
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":121,"threads_started":1}
W20260812 03:03:40.621363 5506 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/krb5cc' of type FILE
Aug 12 03:03:40 dist-test-slave-6bbx krb5kdc[14105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:40.622282 5506 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 03:03:40 dist-test-slave-6bbx krb5kdc[14105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:40.623059 5506 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 03:03:40 dist-test-slave-6bbx krb5kdc[14105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.129@KRBTEST.COM
I20260812 03:03:40.631793 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:40.623809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43598 (local address 127.5.96.129:39595)
0812 03:03:40.623923 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:40.623927 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:40.624007 (+ 80us) server_negotiation.cc:407] Connection header received
0812 03:03:40.624117 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:40.624120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:40.624160 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:40.624229 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:40.624720 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:40.625461 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:40.626255 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:40.626664 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:40.626761 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:40.628808 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:40.628818 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:40.628829 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:40.628851 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:40.630126 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:40.630519 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:40.630524 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:40.630526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:40.630602 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:40.630852 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:40.630856 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:40.630858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:40.631076 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:40.631264 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:40.631464 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 03:03:40.631602 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260812 03:03:40.634804 14966 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:40.634084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43606 (local address 127.5.96.129:39595)
0812 03:03:40.634201 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:40.634204 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:40.634258 (+ 54us) server_negotiation.cc:407] Connection header received
0812 03:03:40.634343 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:40.634345 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:40.634378 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:40.634430 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:40.634722 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43606: BlockingRecv error: recv got EOF from 127.0.0.1:43606 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260812 03:03:40.634913 5506 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/krb5cc' of type FILE
Aug 12 03:03:40 dist-test-slave-6bbx krb5kdc[14105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:40.635774 5506 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 03:03:40 dist-test-slave-6bbx krb5kdc[14105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:40.636497 5506 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 03:03:40 dist-test-slave-6bbx krb5kdc[14105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.129@KRBTEST.COM
I20260812 03:03:40.643442 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:40.637161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43610 (local address 127.5.96.129:39595)
0812 03:03:40.637279 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:40.637281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:40.637296 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:03:40.637386 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:40.637388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:40.637418 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:40.637473 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:40.637844 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:40.638319 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:40.639006 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:40.639202 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:40.639287 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:40.641071 (+ 1784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:40.641074 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:40.641075 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:40.641095 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:40.642277 (+ 1182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:40.642579 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:40.642582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:40.642583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:40.642642 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:40.642810 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:40.642812 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:40.642813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:40.642937 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:40.643015 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:40.643227 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 03:03:40.643313 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260812 03:03:40.645473 14966 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:40.644813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43614 (local address 127.5.96.129:39595)
0812 03:03:40.644928 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:40.644931 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:40.644962 (+ 31us) server_negotiation.cc:407] Connection header received
0812 03:03:40.645049 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:40.645051 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:40.645082 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:40.645137 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:40.645399 (+ 262us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43614: BlockingRecv error: recv got EOF from 127.0.0.1:43614 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260812 03:03:40.645745 5506 init.cc:244] Error while opening credential cache '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786503790637001-5506-0/krb5kdc/krb5cc)
Aug 12 03:03:40 dist-test-slave-6bbx krb5kdc[14105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:40.646543 5506 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 03:03:40 dist-test-slave-6bbx krb5kdc[14105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:40.647282 5506 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 03:03:40 dist-test-slave-6bbx krb5kdc[14105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.129@KRBTEST.COM
I20260812 03:03:40.654003 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:40.647973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43624 (local address 127.5.96.129:39595)
0812 03:03:40.648082 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:40.648085 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:40.648145 (+ 60us) server_negotiation.cc:407] Connection header received
0812 03:03:40.648224 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:40.648226 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:40.648258 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:40.648315 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:40.648685 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:40.649122 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:40.649781 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:40.649964 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:40.650024 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:40.651834 (+ 1810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:40.651837 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:40.651839 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:40.651859 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:40.652849 (+ 990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:40.653184 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:40.653186 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:40.653187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:40.653249 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:40.653434 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:40.653435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:40.653436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:40.653564 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:40.653645 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:40.653799 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 03:03:40.653876 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:03:40.654799 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 14283
I20260812 03:03:40.669919 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 14523
I20260812 03:03:40.682371 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 14760
I20260812 03:03:40.694517 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 14126
2026-08-12T03:03:40Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2211 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:03:40 dist-test-slave-6bbx krb5kdc[14984](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:03:40 dist-test-slave-6bbx krb5kdc[14984](info): set up 2 sockets
Aug 12 03:03:40 dist-test-slave-6bbx krb5kdc[14984](info): commencing operation
krb5kdc: starting...
W20260812 03:03:41.844400 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:03:41 dist-test-slave-6bbx krb5kdc[14984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:03:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:03:41Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.190.keytab.
Entry for principal oryx/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.190.keytab.
I20260812 03:03:41.953981 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:39179
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.190.keytab
--principal=oryx/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:43163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:39179
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:42.060447 15004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:42.060675 15004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:42.060714 15004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:42.064563 15004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:03:42.064607 15004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:42.064627 15004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:42.064656 15004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:03:42.064682 15004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:03:42.069598 15004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:43163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:39179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.190.keytab
--principal=oryx/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:39179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:42.070468 15004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:42.071374 15004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:42.079161 15012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:42.079224 15013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:42.079142 15016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:42.079380 15004 server_base.cc:1034] running on GCE node
I20260812 03:03:42.080029 15004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:42.080989 15004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:42.082136 15004 hybrid_clock.cc:648] HybridClock initialized: now 1786503822082102 us; error 48 us; skew 500 ppm
Aug 12 03:03:42 dist-test-slave-6bbx krb5kdc[14984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503822, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:42.084755 15004 init.cc:377] Logged in from keytab as oryx/127.5.96.190@KRBTEST.COM (short username oryx)
I20260812 03:03:42.085505 15004 webserver.cc:460] Webserver started at http://127.5.96.190:36877/ using document root <none> and password file <none>
I20260812 03:03:42.085986 15004 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:42.086041 15004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:42.086200 15004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:42.088085 15004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "6d88e20979594dfc8777fd21df6c1785"
format_stamp: "Formatted at 2026-08-12 03:03:42 on dist-test-slave-6bbx"
server_key: "f1f0d74f899c97ec6d5816abad168ded"
server_key_iv: "1dbcfc37350a2101a6eafb5902952694"
server_key_version: "encryptionkey@0"
I20260812 03:03:42.088573 15004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "6d88e20979594dfc8777fd21df6c1785"
format_stamp: "Formatted at 2026-08-12 03:03:42 on dist-test-slave-6bbx"
server_key: "f1f0d74f899c97ec6d5816abad168ded"
server_key_iv: "1dbcfc37350a2101a6eafb5902952694"
server_key_version: "encryptionkey@0"
I20260812 03:03:42.091887 15004 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:03:42.093998 15029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:42.095104 15004 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:42.095201 15004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "6d88e20979594dfc8777fd21df6c1785"
format_stamp: "Formatted at 2026-08-12 03:03:42 on dist-test-slave-6bbx"
server_key: "f1f0d74f899c97ec6d5816abad168ded"
server_key_iv: "1dbcfc37350a2101a6eafb5902952694"
server_key_version: "encryptionkey@0"
I20260812 03:03:42.095275 15004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:42.112126 15004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:42.114622 15004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:42.114770 15004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:42.121609 15004 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:39179
I20260812 03:03:42.121925 15113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:39179 every 8 connection(s)
I20260812 03:03:42.122820 15004 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:03:42.125532 15117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:42.128716 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 15004
I20260812 03:03:42.128829 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:03:42.129091 5506 external_mini_cluster.cc:1428] Setting key dbdafd65a3b6bdc647723c81873ca7c7
I20260812 03:03:42.131161 15117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785: Bootstrap starting.
I20260812 03:03:42.133716 15117 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:42.134521 15117 log.cc:824] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:42.136554 15117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785: No bootstrap required, opened a new log
Aug 12 03:03:42 dist-test-slave-6bbx krb5kdc[14984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM
I20260812 03:03:42.140304 15117 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } }
I20260812 03:03:42.140570 15117 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:42.140637 15117 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d88e20979594dfc8777fd21df6c1785, State: Initialized, Role: FOLLOWER
I20260812 03:03:42.141116 15117 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } }
I20260812 03:03:42.141276 15117 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:03:42.141389 15117 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:03:42.141521 15117 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:42.142607 15117 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } }
I20260812 03:03:42.142932 15122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:42.130343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38034 (local address 127.5.96.190:39179)
0812 03:03:42.130726 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:42.130741 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:42.130783 (+ 42us) server_negotiation.cc:407] Connection header received
0812 03:03:42.131405 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:42.131424 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:42.131833 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:42.132195 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:42.133028 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:42.134022 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:42.135101 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:42.135598 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:42.135764 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:42.138313 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:42.138318 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:42.138330 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:42.138355 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:42.139900 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:42.140351 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:42.140355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:42.140359 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:42.140459 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:42.140760 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:42.140764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:42.140765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:42.141247 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:42.141469 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:42.141697 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 03:03:42.141980 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":107,"threads_started":1}
I20260812 03:03:42.142959 15117 leader_election.cc:302] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d88e20979594dfc8777fd21df6c1785; no voters:
I20260812 03:03:42.143253 15117 leader_election.cc:288] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:03:42.143338 15124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:42.143584 15124 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [term 1 LEADER]: Becoming Leader. State: Replica: 6d88e20979594dfc8777fd21df6c1785, State: Running, Role: LEADER
I20260812 03:03:42.143958 15124 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } }
I20260812 03:03:42.144418 15117 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:03:42.145678 15125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d88e20979594dfc8777fd21df6c1785" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } }
I20260812 03:03:42.145818 15125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:42.145678 15126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d88e20979594dfc8777fd21df6c1785. Latest consensus state: current_term: 1 leader_uuid: "6d88e20979594dfc8777fd21df6c1785" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } }
I20260812 03:03:42.145881 15126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:42.146169 15140 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:03:42.149205 15140 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:03:42.154202 15140 catalog_manager.cc:1347] Generated new cluster ID: e62bf77d7cc045c1a93036314667f427
I20260812 03:03:42.154263 15140 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:03:42.166105 15140 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:03:42.166864 15140 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:03:42.173732 15140 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785: Generated new TSK 0
I20260812 03:03:42.174276 15140 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.96.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.96.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.96.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.129.keytab.
Entry for principal oryx/127.5.96.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.129.keytab.
I20260812 03:03:42.200342 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:0
--local_ip_for_outbound_sockets=127.5.96.129
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:39179
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.129.keytab
--principal=oryx/127.5.96.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:43163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:42.331385 15155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:42.331607 15155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:42.331682 15155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:42.335603 15155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:42.335652 15155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:42.335758 15155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:03:42.340685 15155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:43163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.129.keytab
--principal=oryx/127.5.96.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:39179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:42.341583 15155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:42.342525 15155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:42.351616 15170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:42.351624 15163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:42.351733 15155 server_base.cc:1034] running on GCE node
W20260812 03:03:42.351634 15166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:42.352241 15155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:42.352803 15155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:42.353955 15155 hybrid_clock.cc:648] HybridClock initialized: now 1786503822353940 us; error 30 us; skew 500 ppm
Aug 12 03:03:42 dist-test-slave-6bbx krb5kdc[14984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503822, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:42.356513 15155 init.cc:377] Logged in from keytab as oryx/127.5.96.129@KRBTEST.COM (short username oryx)
I20260812 03:03:42.357321 15155 webserver.cc:460] Webserver started at http://127.5.96.129:38821/ using document root <none> and password file <none>
I20260812 03:03:42.357816 15155 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:42.357877 15155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:42.358060 15155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:42.359985 15155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/data/instance:
uuid: "ab340b69c0a246b4bfc2a120a824d8cd"
format_stamp: "Formatted at 2026-08-12 03:03:42 on dist-test-slave-6bbx"
server_key: "221a419ffb9b720d7c3e6d0d448d021e"
server_key_iv: "755f866e8f37f0c03e3f53cf6f59f979"
server_key_version: "encryptionkey@0"
I20260812 03:03:42.360502 15155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance:
uuid: "ab340b69c0a246b4bfc2a120a824d8cd"
format_stamp: "Formatted at 2026-08-12 03:03:42 on dist-test-slave-6bbx"
server_key: "221a419ffb9b720d7c3e6d0d448d021e"
server_key_iv: "755f866e8f37f0c03e3f53cf6f59f979"
server_key_version: "encryptionkey@0"
I20260812 03:03:42.363965 15155 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:03:42.366179 15183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:42.367307 15155 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:42.367399 15155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "ab340b69c0a246b4bfc2a120a824d8cd"
format_stamp: "Formatted at 2026-08-12 03:03:42 on dist-test-slave-6bbx"
server_key: "221a419ffb9b720d7c3e6d0d448d021e"
server_key_iv: "755f866e8f37f0c03e3f53cf6f59f979"
server_key_version: "encryptionkey@0"
I20260812 03:03:42.367476 15155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:42.383430 15155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:42.385942 15155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:42.386085 15155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:42.386567 15155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:42.387285 15155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:42.387331 15155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:42.387365 15155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:42.387385 15155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:42.395366 15155 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:44821
I20260812 03:03:42.395394 15375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:44821 every 8 connection(s)
I20260812 03:03:42.396229 15155 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
Aug 12 03:03:42 dist-test-slave-6bbx krb5kdc[14984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503822, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM
I20260812 03:03:42.406342 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 15155
I20260812 03:03:42.406446 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance
I20260812 03:03:42.406774 5506 external_mini_cluster.cc:1428] Setting key 08306bb5d1b15827561447276ea72834
I20260812 03:03:42.408434 15122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:42.397855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:53013 (local address 127.5.96.190:39179)
0812 03:03:42.397974 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:42.397978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:42.398795 (+ 817us) server_negotiation.cc:407] Connection header received
0812 03:03:42.400120 (+ 1325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:42.400123 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:42.400169 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:42.400228 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:42.401229 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:42.401744 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:42.402784 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:42.402992 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:42.403055 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:42.405169 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:42.405173 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:42.405176 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:42.405202 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:42.406465 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:42.407018 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:42.407021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:42.407022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:42.407098 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:42.407422 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:42.407425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:42.407426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:42.407632 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:42.407745 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:42.408217 (+ 472us) server_negotiation.cc:299] Negotiation successful
0812 03:03:42.408323 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 03:03:42.409343 15379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:42.398211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.5.96.129:53013)
0812 03:03:42.398625 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:42.398666 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:42.399926 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:42.400370 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:42.400376 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:42.400690 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:42.401118 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:42.401131 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:42.401838 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:42.401840 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:42.402690 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:42.402699 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:42.403089 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:42.403092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:42.403218 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:42.403693 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:42.403714 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:42.405002 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:42.406576 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:42.406582 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:42.406596 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:42.406929 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:42.407181 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:42.407183 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:42.407184 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:42.407322 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:42.407749 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:42.407754 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:42.407971 (+ 217us) client_negotiation.cc:769] Sending connection context
0812 03:03:42.408155 (+ 184us) client_negotiation.cc:240] Negotiation successful
0812 03:03:42.408450 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":105,"threads_started":1}
I20260812 03:03:42.410437 15377 heartbeater.cc:347] Connected to a master server at 127.5.96.190:39179
I20260812 03:03:42.410667 15377 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:42.411181 15377 heartbeater.cc:511] Master 127.5.96.190:39179 requested a full tablet report, sending...
I20260812 03:03:42.412647 15058 ts_manager.cc:194] Registered new tserver with Master: ab340b69c0a246b4bfc2a120a824d8cd (127.5.96.129:44821)
I20260812 03:03:42.413769 15058 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.96.129@KRBTEST.COM'} at 127.5.96.129:53013
WARNING: no policy specified for oryx/127.5.96.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.96.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.96.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.130.keytab.
Entry for principal oryx/127.5.96.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.130.keytab.
I20260812 03:03:42.434083 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:0
--local_ip_for_outbound_sockets=127.5.96.130
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:39179
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.130.keytab
--principal=oryx/127.5.96.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:43163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:42.540505 15382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:42.540720 15382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:42.540757 15382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:42.544606 15382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:42.544651 15382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:42.544708 15382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:03:42.549774 15382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:43163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.130.keytab
--principal=oryx/127.5.96.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:39179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:42.550582 15382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:42.551425 15382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:42.560284 15394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:42.560225 15391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:42.560225 15392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:42.560601 15382 server_base.cc:1034] running on GCE node
I20260812 03:03:42.560967 15382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:42.561503 15382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:42.562644 15382 hybrid_clock.cc:648] HybridClock initialized: now 1786503822562620 us; error 39 us; skew 500 ppm
Aug 12 03:03:42 dist-test-slave-6bbx krb5kdc[14984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503822, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:42.565145 15382 init.cc:377] Logged in from keytab as oryx/127.5.96.130@KRBTEST.COM (short username oryx)
I20260812 03:03:42.565932 15382 webserver.cc:460] Webserver started at http://127.5.96.130:43899/ using document root <none> and password file <none>
I20260812 03:03:42.566422 15382 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:42.566480 15382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:42.566664 15382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:42.568531 15382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/data/instance:
uuid: "fbc247aa6b9d4c308db729ea9b3c80a0"
format_stamp: "Formatted at 2026-08-12 03:03:42 on dist-test-slave-6bbx"
server_key: "4adbc5f452ce7b41ccbcfd8606fff969"
server_key_iv: "d1245b68a5a9b358a78d855d24c52e12"
server_key_version: "encryptionkey@0"
I20260812 03:03:42.569036 15382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance:
uuid: "fbc247aa6b9d4c308db729ea9b3c80a0"
format_stamp: "Formatted at 2026-08-12 03:03:42 on dist-test-slave-6bbx"
server_key: "4adbc5f452ce7b41ccbcfd8606fff969"
server_key_iv: "d1245b68a5a9b358a78d855d24c52e12"
server_key_version: "encryptionkey@0"
I20260812 03:03:42.572486 15382 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:03:42.574640 15406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:42.575781 15382 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:03:42.575877 15382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "fbc247aa6b9d4c308db729ea9b3c80a0"
format_stamp: "Formatted at 2026-08-12 03:03:42 on dist-test-slave-6bbx"
server_key: "4adbc5f452ce7b41ccbcfd8606fff969"
server_key_iv: "d1245b68a5a9b358a78d855d24c52e12"
server_key_version: "encryptionkey@0"
I20260812 03:03:42.575968 15382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:42.602922 15382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:42.605418 15382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:42.605576 15382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:42.606096 15382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:42.606854 15382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:42.606900 15382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:42.606941 15382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:42.606971 15382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:42.615146 15382 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:33861
I20260812 03:03:42.615170 15606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:33861 every 8 connection(s)
I20260812 03:03:42.615942 15382 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:03:42.619552 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 15382
I20260812 03:03:42.619658 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance
I20260812 03:03:42.619967 5506 external_mini_cluster.cc:1428] Setting key 60f1efde78e4516be696d7ac2cd5d343
Aug 12 03:03:42 dist-test-slave-6bbx krb5kdc[14984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503822, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM
I20260812 03:03:42.628743 15122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:42.617519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:36923 (local address 127.5.96.190:39179)
0812 03:03:42.617636 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:42.617640 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:42.618406 (+ 766us) server_negotiation.cc:407] Connection header received
0812 03:03:42.619640 (+ 1234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:42.619644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:42.619731 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:42.619819 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:42.621005 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:42.621609 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:42.622704 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:42.622932 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:42.623016 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:42.625491 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:42.625494 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:42.625496 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:42.625519 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:42.626773 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:42.627329 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:42.627331 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:42.627332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:42.627399 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:42.627769 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:42.627771 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:42.627772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:42.627930 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:42.628001 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:42.628541 (+ 540us) server_negotiation.cc:299] Negotiation successful
0812 03:03:42.628630 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:03:42.629796 15610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:42.617859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.5.96.130:36923)
0812 03:03:42.618275 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:42.618315 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:42.619445 (+ 1130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:42.619991 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:42.619999 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:42.620388 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:42.620865 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:42.620878 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:42.621725 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:42.621728 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:42.622585 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:42.622594 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:42.623061 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:42.623065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:42.623217 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:42.623751 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:42.623766 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:42.625338 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:42.626883 (+ 1545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:42.626889 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:42.626903 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:42.627238 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:42.627473 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:42.627477 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:42.627479 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:42.627655 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:42.628023 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:42.628029 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:42.628320 (+ 291us) client_negotiation.cc:769] Sending connection context
0812 03:03:42.628524 (+ 204us) client_negotiation.cc:240] Negotiation successful
0812 03:03:42.628845 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":116,"threads_started":1}
WARNING: no policy specified for oryx/127.5.96.131@KRBTEST.COM; defaulting to no policy
I20260812 03:03:42.630898 15608 heartbeater.cc:347] Connected to a master server at 127.5.96.190:39179
I20260812 03:03:42.631108 15608 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:42.631572 15608 heartbeater.cc:511] Master 127.5.96.190:39179 requested a full tablet report, sending...
I20260812 03:03:42.632616 15058 ts_manager.cc:194] Registered new tserver with Master: fbc247aa6b9d4c308db729ea9b3c80a0 (127.5.96.130:33861)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.96.131@KRBTEST.COM" created.
I20260812 03:03:42.633203 15058 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.96.130@KRBTEST.COM'} at 127.5.96.130:36923
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.96.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.131.keytab.
Entry for principal oryx/127.5.96.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.131.keytab.
I20260812 03:03:42.645462 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:0
--local_ip_for_outbound_sockets=127.5.96.131
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:39179
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.131.keytab
--principal=oryx/127.5.96.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:43163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:42.753400 15615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:42.753616 15615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:42.753654 15615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:42.757599 15615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:42.757649 15615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:42.757702 15615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:03:42.762660 15615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:43163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.131.keytab
--principal=oryx/127.5.96.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:39179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:42.763494 15615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:42.764400 15615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:42.773340 15627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:42.773331 15623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:42.773332 15624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:42.773569 15615 server_base.cc:1034] running on GCE node
I20260812 03:03:42.774026 15615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:42.774624 15615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:42.775739 15615 hybrid_clock.cc:648] HybridClock initialized: now 1786503822775715 us; error 28 us; skew 500 ppm
Aug 12 03:03:42 dist-test-slave-6bbx krb5kdc[14984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503822, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:42.778303 15615 init.cc:377] Logged in from keytab as oryx/127.5.96.131@KRBTEST.COM (short username oryx)
I20260812 03:03:42.779132 15615 webserver.cc:460] Webserver started at http://127.5.96.131:41877/ using document root <none> and password file <none>
I20260812 03:03:42.779714 15615 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:42.779788 15615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:42.779982 15615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:42.781854 15615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/data/instance:
uuid: "e19a80e5310d4f2396df01f7d289dc05"
format_stamp: "Formatted at 2026-08-12 03:03:42 on dist-test-slave-6bbx"
server_key: "3def5d954e1043456911e6c854335b42"
server_key_iv: "8634de2bce991dc6afa0849ba3b33cd6"
server_key_version: "encryptionkey@0"
I20260812 03:03:42.782385 15615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance:
uuid: "e19a80e5310d4f2396df01f7d289dc05"
format_stamp: "Formatted at 2026-08-12 03:03:42 on dist-test-slave-6bbx"
server_key: "3def5d954e1043456911e6c854335b42"
server_key_iv: "8634de2bce991dc6afa0849ba3b33cd6"
server_key_version: "encryptionkey@0"
I20260812 03:03:42.785861 15615 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 03:03:42.788089 15640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:42.789155 15615 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:03:42.789273 15615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "e19a80e5310d4f2396df01f7d289dc05"
format_stamp: "Formatted at 2026-08-12 03:03:42 on dist-test-slave-6bbx"
server_key: "3def5d954e1043456911e6c854335b42"
server_key_iv: "8634de2bce991dc6afa0849ba3b33cd6"
server_key_version: "encryptionkey@0"
I20260812 03:03:42.789361 15615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:42.808316 15615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:42.810690 15615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:42.810844 15615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:42.811355 15615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:42.812139 15615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:42.812184 15615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:42.812233 15615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:42.812269 15615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:42.820374 15615 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:43647
I20260812 03:03:42.820405 15828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:43647 every 8 connection(s)
I20260812 03:03:42.821141 15615 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:03:42 dist-test-slave-6bbx krb5kdc[14984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503822, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM
I20260812 03:03:42.830878 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 15615
I20260812 03:03:42.830984 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance
I20260812 03:03:42.831254 5506 external_mini_cluster.cc:1428] Setting key 17c577bf643a696f433bcce27e197168
I20260812 03:03:42.833415 15122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:42.822781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:58559 (local address 127.5.96.190:39179)
0812 03:03:42.822927 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:42.822932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:42.823698 (+ 766us) server_negotiation.cc:407] Connection header received
0812 03:03:42.824926 (+ 1228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:42.824928 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:42.824977 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:42.825044 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:42.826057 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:42.826571 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:42.827575 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:42.827794 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:42.827857 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:42.829989 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:42.829992 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:42.829994 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:42.830018 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:42.831393 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:42.831920 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:42.831924 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:42.831926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:42.832017 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:42.832352 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:42.832354 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:42.832356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:42.832566 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:42.832659 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:42.833128 (+ 469us) server_negotiation.cc:299] Negotiation successful
0812 03:03:42.833259 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:03:42.834087 15836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:42.823128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.5.96.131:58559)
0812 03:03:42.823552 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:42.823591 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:42.824737 (+ 1146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:42.825177 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:42.825184 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:42.825502 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:42.825943 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:42.825956 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:42.826654 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:42.826656 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:42.827487 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:42.827496 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:42.827906 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:42.827909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:42.828045 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:42.828485 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:42.828500 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:42.829836 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:42.831490 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:42.831496 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:42.831509 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:42.831843 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:42.832110 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:42.832114 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:42.832116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:42.832256 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:42.832678 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:42.832684 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:42.832907 (+ 223us) client_negotiation.cc:769] Sending connection context
0812 03:03:42.833063 (+ 156us) client_negotiation.cc:240] Negotiation successful
0812 03:03:42.833355 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":121,"threads_started":1}
I20260812 03:03:42.835058 15831 heartbeater.cc:347] Connected to a master server at 127.5.96.190:39179
I20260812 03:03:42.835268 15831 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:42.835731 15831 heartbeater.cc:511] Master 127.5.96.190:39179 requested a full tablet report, sending...
I20260812 03:03:42.836637 15058 ts_manager.cc:194] Registered new tserver with Master: e19a80e5310d4f2396df01f7d289dc05 (127.5.96.131:43647)
I20260812 03:03:42.837149 15058 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.96.131@KRBTEST.COM'} at 127.5.96.131:58559
I20260812 03:03:42.844662 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.5.96.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.96.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.96.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.189.keytab.
Entry for principal oryx/127.5.96.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.189.keytab.
I20260812 03:03:42.868254 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.189:44711
--webserver_interface=127.5.96.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.189.keytab
--principal=oryx/127.5.96.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:43163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:39179,127.5.96.189:44711
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:42.975276 15846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:42.975495 15846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:42.975533 15846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:42.979357 15846 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:03:42.979403 15846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:42.979421 15846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:42.979440 15846 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:03:42.979458 15846 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:03:42.984442 15846 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:43163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:39179,127.5.96.189:44711
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.189.keytab
--principal=oryx/127.5.96.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.189:44711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.96.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
W20260812 03:03:42.984601 15846 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.5.96.190:39179,127.5.96.189:44711'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 03:03:42.985330 15846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:42.986227 15846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:42.993956 15854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:42.994165 15853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:42.994177 15846 server_base.cc:1034] running on GCE node
W20260812 03:03:42.993969 15857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:42.994789 15846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:42.995800 15846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:42.996954 15846 hybrid_clock.cc:648] HybridClock initialized: now 1786503822996937 us; error 25 us; skew 500 ppm
Aug 12 03:03:42 dist-test-slave-6bbx krb5kdc[14984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503822, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:42.999573 15846 init.cc:377] Logged in from keytab as oryx/127.5.96.189@KRBTEST.COM (short username oryx)
I20260812 03:03:43.000382 15846 webserver.cc:460] Webserver started at http://127.5.96.189:33125/ using document root <none> and password file <none>
I20260812 03:03:43.000888 15846 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:43.000954 15846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:43.001139 15846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:43.003026 15846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-1/data/instance:
uuid: "985406e5f9d44f50b81a6191e3026349"
format_stamp: "Formatted at 2026-08-12 03:03:42 on dist-test-slave-6bbx"
server_key: "260f48b9273b0431b4a74329134565be"
server_key_iv: "25d376252f0a054f105bca2fc1591d4e"
server_key_version: "encryptionkey@0"
I20260812 03:03:43.003542 15846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-1/wal/instance:
uuid: "985406e5f9d44f50b81a6191e3026349"
format_stamp: "Formatted at 2026-08-12 03:03:42 on dist-test-slave-6bbx"
server_key: "260f48b9273b0431b4a74329134565be"
server_key_iv: "25d376252f0a054f105bca2fc1591d4e"
server_key_version: "encryptionkey@0"
I20260812 03:03:43.006997 15846 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:03:43.009235 15867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:43.010313 15846 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:03:43.010423 15846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-1/wal
uuid: "985406e5f9d44f50b81a6191e3026349"
format_stamp: "Formatted at 2026-08-12 03:03:42 on dist-test-slave-6bbx"
server_key: "260f48b9273b0431b4a74329134565be"
server_key_iv: "25d376252f0a054f105bca2fc1591d4e"
server_key_version: "encryptionkey@0"
I20260812 03:03:43.010509 15846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:43.022814 15846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:43.025204 15846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:43.025362 15846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:43.031693 15846 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.189:44711
I20260812 03:03:43.031953 15951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.189:44711 every 8 connection(s)
I20260812 03:03:43.032528 15846 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-1/data/info.pb
I20260812 03:03:43.033190 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 15846
I20260812 03:03:43.035483 15953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:43.039464 15953 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:03:43 dist-test-slave-6bbx krb5kdc[14984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.96.189@KRBTEST.COM
Aug 12 03:03:43 dist-test-slave-6bbx krb5kdc[14984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503822, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.189@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM
I20260812 03:03:43.047178 15959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.034671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54048 (local address 127.5.96.189:44711)
0812 03:03:43.035029 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:43.035044 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:43.035079 (+ 35us) server_negotiation.cc:407] Connection header received
0812 03:03:43.035760 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:43.035780 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:43.036084 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:43.036347 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:43.037197 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.038145 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.039259 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.039739 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.039880 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.042474 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:43.042479 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:43.042488 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:43.042515 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:43.044299 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.044754 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.044759 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.044763 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.044870 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.045161 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.045165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.045167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.045702 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:43.045870 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:43.046132 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 03:03:43.046382 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":102,"threads_started":1}
I20260812 03:03:43.050064 15960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.041211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.0.0.1:38042)
0812 03:03:43.041521 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:43.041567 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:43.041690 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:43.041986 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:43.041991 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:43.042334 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:43.042456 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.042462 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.043358 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.043362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.044383 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.044391 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.044936 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.044940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.045043 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.045484 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:43.045498 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:43.046994 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:43.048585 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.048592 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.048596 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.048867 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.049157 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.049160 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.049162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.049255 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.049613 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:43.049617 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:43.049703 (+ 86us) client_negotiation.cc:769] Sending connection context
0812 03:03:43.049815 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 03:03:43.049936 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":77,"threads_started":1}
I20260812 03:03:43.050066 15122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.041298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38042 (local address 127.5.96.190:39179)
0812 03:03:43.041450 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:43.041454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:43.041643 (+ 189us) server_negotiation.cc:407] Connection header received
0812 03:03:43.041798 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:43.041802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:43.041851 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:43.041927 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:43.042583 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.043237 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.044487 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.044834 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.044911 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.047122 (+ 2211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:43.047124 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:43.047126 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:43.047149 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:43.048490 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.048970 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.048973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.048975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.049067 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.049328 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.049329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.049331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.049539 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:43.049613 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:43.049865 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 03:03:43.049956 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 03:03:43.051095 15953 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:03:43 dist-test-slave-6bbx krb5kdc[14984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503822, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.189@KRBTEST.COM for oryx/127.5.96.189@KRBTEST.COM
I20260812 03:03:43.057623 15960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.051488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:44711 (local address 127.0.0.1:54060)
0812 03:03:43.051583 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:43.051595 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:43.051659 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:43.051865 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:43.051867 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:43.051980 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:43.052064 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.052068 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.053085 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.053087 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.053568 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.053574 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.053920 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.053922 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.053989 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.054301 (+ 312us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:43.054311 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:43.055288 (+ 977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:43.056593 (+ 1305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.056595 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.056596 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.056777 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.056965 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.056967 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.056968 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.057029 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.057278 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:43.057279 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:43.057342 (+ 63us) client_negotiation.cc:769] Sending connection context
0812 03:03:43.057410 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 03:03:43.057500 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 03:03:43.057686 15959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.051561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54060 (local address 127.5.96.189:44711)
0812 03:03:43.051658 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:43.051677 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:43.051691 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:03:43.051751 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:43.051754 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:43.051787 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:43.051841 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:43.052138 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.052994 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.053642 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.053851 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.053914 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.055400 (+ 1486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:43.055402 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:43.055404 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:43.055424 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:43.056506 (+ 1082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.056848 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.056850 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.056852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.056911 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.057084 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.057085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.057086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.057215 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:43.057288 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:43.057448 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 03:03:43.057549 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":39}
I20260812 03:03:43.060123 15953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349: Bootstrap starting.
I20260812 03:03:43.062224 15953 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:43.062886 15953 log.cc:824] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:43.064564 15953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349: No bootstrap required, opened a new log
I20260812 03:03:43.067303 15953 raft_consensus.cc:348] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } }
I20260812 03:03:43.067476 15953 raft_consensus.cc:374] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:43.067528 15953 raft_consensus.cc:729] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 985406e5f9d44f50b81a6191e3026349, State: Initialized, Role: FOLLOWER
I20260812 03:03:43.067942 15953 consensus_queue.cc:260] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } }
I20260812 03:03:43.068362 15962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } } }
I20260812 03:03:43.068476 15962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:03:43.069013 15953 sys_catalog.cc:564] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 03:03:43.073187 15980 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:03:43.073261 15980 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:03:43.076148 15846 master_runner.cc:428] Detected that this master 985406e5f9d44f50b81a6191e3026349 is joining an existing cluster
I20260812 03:03:43.076198 15846 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.96.189:44711
I20260812 03:03:43.085307 15995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.078314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:44711 (local address 127.0.0.1:54072)
0812 03:03:43.078687 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:43.078701 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:43.078840 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:43.079065 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:43.079096 (+ 31us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:43.079316 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:43.079439 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.079452 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.080386 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.080389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.081004 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.081013 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.081453 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.081457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.081536 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.081959 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:43.081973 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:43.082332 (+ 359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:43.083863 (+ 1531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.083866 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.083868 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.084079 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.084303 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.084305 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.084307 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.084391 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.084858 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:43.084860 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:43.084939 (+ 79us) client_negotiation.cc:769] Sending connection context
0812 03:03:43.085018 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 03:03:43.085131 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":85,"threads_started":1}
I20260812 03:03:43.086046 15959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.078423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54072 (local address 127.5.96.189:44711)
0812 03:03:43.078565 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:43.078569 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:43.078782 (+ 213us) server_negotiation.cc:407] Connection header received
0812 03:03:43.078938 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:43.078941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:43.078978 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:43.079053 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:43.079542 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.080260 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.081106 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.081346 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.081429 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.082445 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:43.082449 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:43.082452 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:43.082474 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:43.083762 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.084162 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.084165 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.084167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.084231 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.084460 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.084463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.084465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.084742 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:43.084829 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:43.085811 (+ 982us) server_negotiation.cc:299] Negotiation successful
0812 03:03:43.085913 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":59}
I20260812 03:03:43.095772 15122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.080112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38054 (local address 127.5.96.190:39179)
0812 03:03:43.080230 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:43.080234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:43.083931 (+ 3697us) server_negotiation.cc:407] Connection header received
0812 03:03:43.084095 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:43.084099 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:43.084146 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:43.084195 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:43.087968 (+ 3773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.088772 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.089956 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.090301 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.090369 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.092420 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:43.092424 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:43.092427 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:43.092452 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:43.094053 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.094506 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.094509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.094511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.094590 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.094876 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.094879 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.094881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.095078 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:43.095153 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:43.095461 (+ 308us) server_negotiation.cc:299] Negotiation successful
0812 03:03:43.095572 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 03:03:43.096222 15996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.080248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.0.0.1:38054)
0812 03:03:43.083841 (+ 3593us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:43.083855 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:43.083994 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:43.084339 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:43.084341 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:43.084545 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:43.087830 (+ 3285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.087838 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.088908 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.088911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.089822 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.089831 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.090518 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.090521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.090613 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.091803 (+ 1190us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:43.091817 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:43.092273 (+ 456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:43.094170 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.094174 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.094176 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.094404 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.094687 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.094690 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.094692 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.094782 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.095205 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:43.095208 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:43.095293 (+ 85us) client_negotiation.cc:769] Sending connection context
0812 03:03:43.095906 (+ 613us) client_negotiation.cc:240] Negotiation successful
0812 03:03:43.096049 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3528,"mutex_wait_us":3136,"thread_start_us":84,"threads_started":1}
Aug 12 03:03:43 dist-test-slave-6bbx krb5kdc[14984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503822, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.190@KRBTEST.COM for oryx/127.5.96.189@KRBTEST.COM
WARNING: no policy specified for oryx/127.5.96.188@KRBTEST.COM; defaulting to no policy
I20260812 03:03:43.110155 15959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.101322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54076 (local address 127.5.96.189:44711)
0812 03:03:43.101411 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:43.101416 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:43.101428 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:03:43.101489 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:43.101492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:43.101536 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:43.101609 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:43.102351 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.103091 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.104239 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.104442 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.104519 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.106818 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:43.106821 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:43.106823 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:43.106844 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:43.108297 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.108850 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.108853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.108855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.108939 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.109217 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.109228 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.109229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.109423 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:43.109516 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:43.109900 (+ 384us) server_negotiation.cc:299] Negotiation successful
0812 03:03:43.110007 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 03:03:43.110213 15999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.100619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:44711 (local address 127.0.0.1:54076)
0812 03:03:43.100979 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:43.101032 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:43.101148 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:43.101653 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:43.101659 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:43.102081 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:43.102219 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.102226 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.103224 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.103227 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.104122 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.104134 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.104549 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.104553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.104656 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.105085 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:43.105099 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:43.106687 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:43.108423 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.108433 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.108440 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.108744 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.109033 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.109037 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.109039 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.109130 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.109537 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:43.109544 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:43.109682 (+ 138us) client_negotiation.cc:769] Sending connection context
0812 03:03:43.109847 (+ 165us) client_negotiation.cc:240] Negotiation successful
0812 03:03:43.110036 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":89,"threads_started":1}
I20260812 03:03:43.111570 15058 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.96.189:44711: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "985406e5f9d44f50b81a6191e3026349"
member_type: NON_VOTER
last_known_addr {
host: "127.5.96.189"
port: 44711
}
attrs {
promote: true
}
}
dest_uuid: "6d88e20979594dfc8777fd21df6c1785"
cas_config_opid_index: -1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.96.188@KRBTEST.COM" created.
I20260812 03:03:43.112427 15058 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: NON_VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: true } }
I20260812 03:03:43.114370 16003 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 985406e5f9d44f50b81a6191e3026349 (127.5.96.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: NON_VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: true } } }
I20260812 03:03:43.114989 15914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:43.116009 16003 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.96.189:44711
I20260812 03:03:43.116123 15914 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [term 1 FOLLOWER]: Refusing update from remote peer 6d88e20979594dfc8777fd21df6c1785: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 03:03:43.116484 15846 master.cc:537] Master@127.5.96.189:44711 shutting down...
I20260812 03:03:43.117722 16001 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: NON_VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 03:03:43.120012 15035 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260812 03:03:43.120668 16007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6d88e20979594dfc8777fd21df6c1785" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: NON_VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: true } } }
I20260812 03:03:43.120781 16007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [sys.catalog]: This master's current role is: LEADER
W20260812 03:03:43.125809 15035 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 -> Peer 985406e5f9d44f50b81a6191e3026349 (127.5.96.189:44711): Couldn't send request to peer 985406e5f9d44f50b81a6191e3026349. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 03:03:43.126343 15846 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:43.126609 15846 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:43.126768 15846 tablet_replica.cc:330] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.96.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.188.keytab.
Entry for principal oryx/127.5.96.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.188.keytab.
I20260812 03:03:43.132213 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.188:38451
--webserver_interface=127.5.96.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.188.keytab
--principal=oryx/127.5.96.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:43163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:39179,127.5.96.189:44711,127.5.96.188:38451
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:03:43.141970 15846 master.cc:552] Master@127.5.96.189:44711 shutdown complete.
I20260812 03:03:43.142017 15846 master_runner.cc:305] Clearing existing system tablet
I20260812 03:03:43.142251 15846 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:43.142302 15846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:43.143324 15846 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:03:43.144127 16015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:43.144331 15846 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:43.144536 15846 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 03:03:43.144577 15846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-1/wal
uuid: "985406e5f9d44f50b81a6191e3026349"
format_stamp: "Formatted at 2026-08-12 03:03:42 on dist-test-slave-6bbx"
server_key: "260f48b9273b0431b4a74329134565be"
server_key_iv: "25d376252f0a054f105bca2fc1591d4e"
server_key_version: "encryptionkey@0"
I20260812 03:03:43.146505 15846 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:43.148466 15846 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 03:03:43.148546 15846 log.cc:1196] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 03:03:43.148784 15846 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349: Deleting consensus metadata
I20260812 03:03:43.148939 15846 master_runner.cc:315] Copying system tablet from 127.5.96.190:39179
I20260812 03:03:43.150081 15846 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349: tablet copy: Beginning tablet copy session from remote peer at address 127.5.96.190:39179
I20260812 03:03:43.157253 15122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.150703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38064 (local address 127.5.96.190:39179)
0812 03:03:43.150832 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:43.150837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:43.151007 (+ 170us) server_negotiation.cc:407] Connection header received
0812 03:03:43.151174 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:43.151178 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:43.151221 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:43.151287 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:43.151734 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.152258 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.153078 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.153306 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.153397 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.154440 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:43.154443 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:43.154446 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:43.154473 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:43.155779 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.156166 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.156169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.156171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.156245 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.156487 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.156491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.156492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.156663 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:43.156756 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:43.157000 (+ 244us) server_negotiation.cc:299] Negotiation successful
0812 03:03:43.157103 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:03:43.157275 16022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.150644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.0.0.1:38064)
0812 03:03:43.150922 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:43.150937 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:43.151079 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:43.151314 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:43.151317 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:43.151506 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:43.151604 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.151609 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.152363 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.152366 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.152968 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.152976 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.153410 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.153413 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.153507 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.153980 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:43.153994 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:43.154307 (+ 313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:43.155884 (+ 1577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.155887 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.155889 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.156080 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.156328 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.156330 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.156332 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.156410 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.156751 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:43.156754 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:43.156850 (+ 96us) client_negotiation.cc:769] Sending connection context
0812 03:03:43.156933 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 03:03:43.157071 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":81,"threads_started":1}
I20260812 03:03:43.158036 15086 tablet_copy_service.cc:140] P 6d88e20979594dfc8777fd21df6c1785: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 985406e5f9d44f50b81a6191e3026349 ({username='oryx', principal='oryx/127.5.96.189@KRBTEST.COM'} at 127.0.0.1:38064)
I20260812 03:03:43.158190 15086 tablet_copy_service.cc:161] P 6d88e20979594dfc8777fd21df6c1785: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 985406e5f9d44f50b81a6191e3026349 at {username='oryx', principal='oryx/127.5.96.189@KRBTEST.COM'} at 127.0.0.1:38064: session id = 985406e5f9d44f50b81a6191e3026349-00000000000000000000000000000000
I20260812 03:03:43.159605 15086 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 03:03:43.161100 15846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 03:03:43.162788 15846 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349: tablet copy: Starting download of 0 data blocks...
I20260812 03:03:43.162953 15846 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349: tablet copy: Starting download of 1 WAL segments...
I20260812 03:03:43.164475 15846 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 03:03:43.165807 15086 tablet_copy_service.cc:342] P 6d88e20979594dfc8777fd21df6c1785: Request end of tablet copy session 985406e5f9d44f50b81a6191e3026349-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.96.189@KRBTEST.COM'} at 127.0.0.1:38064
I20260812 03:03:43.165892 15086 tablet_copy_service.cc:433] P 6d88e20979594dfc8777fd21df6c1785: ending tablet copy session 985406e5f9d44f50b81a6191e3026349-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 985406e5f9d44f50b81a6191e3026349
I20260812 03:03:43.172535 15846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:43.173929 16023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:43.173961 16024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:43.174134 16027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:43.174423 15846 server_base.cc:1034] running on GCE node
I20260812 03:03:43.174590 15846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:43.174856 15846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:43.175951 15846 hybrid_clock.cc:648] HybridClock initialized: now 1786503823175942 us; error 23 us; skew 500 ppm
Aug 12 03:03:43 dist-test-slave-6bbx krb5kdc[14984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503823, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:43.177314 15846 init.cc:377] Logged in from keytab as oryx/127.5.96.189@KRBTEST.COM (short username oryx)
I20260812 03:03:43.177717 15846 webserver.cc:460] Webserver started at http://127.5.96.189:35833/ using document root <none> and password file <none>
I20260812 03:03:43.177897 15846 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:43.177932 15846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:43.178962 15846 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:03:43.179795 16043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:43.180018 15846 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:03:43.180086 15846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-1/wal
uuid: "985406e5f9d44f50b81a6191e3026349"
format_stamp: "Formatted at 2026-08-12 03:03:42 on dist-test-slave-6bbx"
server_key: "260f48b9273b0431b4a74329134565be"
server_key_iv: "25d376252f0a054f105bca2fc1591d4e"
server_key_version: "encryptionkey@0"
I20260812 03:03:43.180160 15846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:43.189087 15846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:43.190904 15846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:43.195320 15846 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.189:44711
I20260812 03:03:43.195935 15846 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-1/data/info.pb
I20260812 03:03:43.197237 16130 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:03:43.198037 16130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349: Bootstrap starting.
I20260812 03:03:43.203811 16126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.189:44711 every 8 connection(s)
I20260812 03:03:43.206142 16130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:03:43.206395 16130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349: Bootstrap complete.
I20260812 03:03:43.206830 16130 raft_consensus.cc:348] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: NON_VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: true } }
I20260812 03:03:43.206938 16130 raft_consensus.cc:729] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 985406e5f9d44f50b81a6191e3026349, State: Initialized, Role: LEARNER
I20260812 03:03:43.207044 16130 consensus_queue.cc:260] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: NON_VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: true } }
I20260812 03:03:43.207329 16134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: NON_VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: true } } }
I20260812 03:03:43.207464 16130 sys_catalog.cc:564] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:03:43.207481 16134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [sys.catalog]: This master's current role is: LEARNER
I20260812 03:03:43.210340 16150 catalog_manager.cc:1259] Loaded cluster ID: e62bf77d7cc045c1a93036314667f427
I20260812 03:03:43.210383 16150 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349: loading cluster ID for follower catalog manager: success
I20260812 03:03:43.211859 16150 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349: acquiring CA information for follower catalog manager: success
I20260812 03:03:43.212862 16150 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 03:03:43.282156 16011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:43.282387 16011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:43.282429 16011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:43.286301 16011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:03:43.286347 16011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:43.286366 16011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:43.286396 16011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:03:43.286422 16011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:03:43.291400 16011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:43163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:39179,127.5.96.189:44711,127.5.96.188:38451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.188.keytab
--principal=oryx/127.5.96.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.188:38451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.96.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:43.292349 16011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:43.293295 16011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:43.301081 16166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:43.301075 16168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:43.301201 16011 server_base.cc:1034] running on GCE node
W20260812 03:03:43.301434 16163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:43.301939 16011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:43.302892 16011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:43.304032 16011 hybrid_clock.cc:648] HybridClock initialized: now 1786503823304008 us; error 35 us; skew 500 ppm
Aug 12 03:03:43 dist-test-slave-6bbx krb5kdc[14984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503823, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:43.306511 16011 init.cc:377] Logged in from keytab as oryx/127.5.96.188@KRBTEST.COM (short username oryx)
I20260812 03:03:43.307308 16011 webserver.cc:460] Webserver started at http://127.5.96.188:40403/ using document root <none> and password file <none>
I20260812 03:03:43.307832 16011 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:43.307895 16011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:43.308081 16011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:43.310030 16011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-2/data/instance:
uuid: "5137614d58be42dbadab04e8e76d1a54"
format_stamp: "Formatted at 2026-08-12 03:03:43 on dist-test-slave-6bbx"
server_key: "83b2c0322f4938cd7b2baf5f088de078"
server_key_iv: "d791079d382ae3b0927430d9ca1b0e71"
server_key_version: "encryptionkey@0"
I20260812 03:03:43.310549 16011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-2/wal/instance:
uuid: "5137614d58be42dbadab04e8e76d1a54"
format_stamp: "Formatted at 2026-08-12 03:03:43 on dist-test-slave-6bbx"
server_key: "83b2c0322f4938cd7b2baf5f088de078"
server_key_iv: "d791079d382ae3b0927430d9ca1b0e71"
server_key_version: "encryptionkey@0"
I20260812 03:03:43.313936 16011 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260812 03:03:43.316087 16177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:43.317168 16011 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:43.317272 16011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-2/wal
uuid: "5137614d58be42dbadab04e8e76d1a54"
format_stamp: "Formatted at 2026-08-12 03:03:43 on dist-test-slave-6bbx"
server_key: "83b2c0322f4938cd7b2baf5f088de078"
server_key_iv: "d791079d382ae3b0927430d9ca1b0e71"
server_key_version: "encryptionkey@0"
I20260812 03:03:43.317348 16011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:43.339453 16011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:43.341904 16011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:43.342039 16011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:43.348716 16011 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.188:38451
I20260812 03:03:43.348963 16273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.188:38451 every 8 connection(s)
I20260812 03:03:43.349573 16011 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-2/data/info.pb
I20260812 03:03:43.349983 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 16011
I20260812 03:03:43.352337 16277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:43.356426 16277 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:03:43 dist-test-slave-6bbx krb5kdc[14984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.96.188@KRBTEST.COM
Aug 12 03:03:43 dist-test-slave-6bbx krb5kdc[14984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503823, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.188@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM
I20260812 03:03:43.363444 16279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.351331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50012 (local address 127.5.96.188:38451)
0812 03:03:43.351738 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:43.351756 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:43.351792 (+ 36us) server_negotiation.cc:407] Connection header received
0812 03:03:43.352403 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:43.352425 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:43.352753 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:43.353011 (+ 258us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:43.353878 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.354830 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.355984 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.356496 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.356609 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.359358 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:43.359363 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:43.359371 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:43.359397 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:43.360892 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.361294 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.361298 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.361302 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.361408 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.361667 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.361670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.361672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.362023 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:43.362212 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:43.362484 (+ 272us) server_negotiation.cc:299] Negotiation successful
0812 03:03:43.362726 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":112,"threads_started":1}
I20260812 03:03:43.366487 15122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.358346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38078 (local address 127.5.96.190:39179)
0812 03:03:43.358459 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:43.358464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:43.358753 (+ 289us) server_negotiation.cc:407] Connection header received
0812 03:03:43.358888 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:43.358891 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:43.358938 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:43.359002 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:43.359621 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.360176 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.361029 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.361257 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.361343 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.363316 (+ 1973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:43.363319 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:43.363322 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:43.363350 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:43.365171 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.365570 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.365572 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.365574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.365644 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.365842 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.365844 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.365845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.365990 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:43.366057 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:43.366297 (+ 240us) server_negotiation.cc:299] Negotiation successful
0812 03:03:43.366381 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 03:03:43.366504 16280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.358193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.0.0.1:38078)
0812 03:03:43.358641 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:43.358677 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:43.358798 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:43.359059 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:43.359065 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:43.359383 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:43.359497 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.359503 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.360280 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.360283 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.360918 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.360927 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.361355 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.361359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.361466 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.361882 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:43.361893 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:43.363180 (+ 1287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:43.365279 (+ 2099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.365284 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.365289 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.365493 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.365703 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.365705 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.365706 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.365784 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.366056 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:43.366060 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:43.366147 (+ 87us) client_negotiation.cc:769] Sending connection context
0812 03:03:43.366251 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 03:03:43.366374 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":120,"threads_started":1}
I20260812 03:03:43.367496 16277 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:03:43 dist-test-slave-6bbx krb5kdc[14984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503823, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.188@KRBTEST.COM for oryx/127.5.96.189@KRBTEST.COM
I20260812 03:03:43.374729 16280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.367934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:44711 (local address 127.0.0.1:54078)
0812 03:03:43.368711 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:43.368730 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:43.368798 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:43.368993 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:43.368995 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:43.369109 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:43.369186 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.369190 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.369973 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.369975 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.370489 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.370496 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.370935 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.370938 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.371006 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.371325 (+ 319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:43.371335 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:43.372355 (+ 1020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:43.373682 (+ 1327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.373684 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.373685 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.373873 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.374074 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.374075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.374077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.374140 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.374402 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:43.374404 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:43.374469 (+ 65us) client_negotiation.cc:769] Sending connection context
0812 03:03:43.374536 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 03:03:43.374622 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718}
I20260812 03:03:43.374790 16281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.368043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54078 (local address 127.5.96.189:44711)
0812 03:03:43.368276 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:43.368289 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:43.368780 (+ 491us) server_negotiation.cc:407] Connection header received
0812 03:03:43.368873 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:43.368875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:43.368915 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:43.368978 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:43.369264 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.369898 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.370580 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.370830 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.370920 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.372484 (+ 1564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:43.372488 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:43.372489 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:43.372511 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:43.373599 (+ 1088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.373944 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.373946 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.373947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.374012 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.374196 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.374198 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.374199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.374343 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:43.374410 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:43.374576 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 03:03:43.374664 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":90,"threads_started":1}
I20260812 03:03:43.375428 16277 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.188" port: 38451 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:03:43 dist-test-slave-6bbx krb5kdc[14984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503823, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.188@KRBTEST.COM for oryx/127.5.96.188@KRBTEST.COM
I20260812 03:03:43.382014 16280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.375849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:38451 (local address 127.0.0.1:50024)
0812 03:03:43.375937 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:43.375949 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:43.376012 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:43.376163 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:43.376165 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:43.376284 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:43.376346 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.376350 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.377282 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.377284 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.377740 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.377745 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.378090 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.378093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.378148 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.378448 (+ 300us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:43.378457 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:43.379550 (+ 1093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:43.380910 (+ 1360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.380912 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.380914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.381085 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.381304 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.381306 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.381307 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.381375 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.381669 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:43.381670 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:43.381739 (+ 69us) client_negotiation.cc:769] Sending connection context
0812 03:03:43.381814 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 03:03:43.381897 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 03:03:43.382102 16279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.375896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50024 (local address 127.5.96.188:38451)
0812 03:03:43.375987 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:43.375989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:43.376001 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:03:43.376066 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:43.376068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:43.376097 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:43.376148 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:43.376416 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.377194 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.377814 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.378022 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.378084 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.379694 (+ 1610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:43.379699 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:43.379701 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:43.379733 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:43.380818 (+ 1085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.381154 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.381156 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.381157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.381217 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.381445 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.381448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.381449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.381601 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:43.381671 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:43.381854 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 03:03:43.381949 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":9,"server-negotiator.queue_time_us":38}
I20260812 03:03:43.384544 16277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54: Bootstrap starting.
I20260812 03:03:43.386688 16277 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:43.387485 16277 log.cc:824] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:43.389201 16277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54: No bootstrap required, opened a new log
I20260812 03:03:43.392159 16277 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 38451 } }
I20260812 03:03:43.392357 16277 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:43.392414 16277 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5137614d58be42dbadab04e8e76d1a54, State: Initialized, Role: FOLLOWER
I20260812 03:03:43.392810 16277 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 38451 } }
I20260812 03:03:43.393301 16283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 38451 } } }
I20260812 03:03:43.393410 16283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:03:43.394009 16277 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 03:03:43.398607 16301 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:03:43.398674 16301 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:03:43.401273 16011 master_runner.cc:428] Detected that this master 5137614d58be42dbadab04e8e76d1a54 is joining an existing cluster
I20260812 03:03:43.401319 16011 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.96.188:38451
I20260812 03:03:43.411516 16310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.403138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.0.0.1:38088)
0812 03:03:43.403416 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:43.403428 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:43.403516 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:43.403764 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:43.403768 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:43.404002 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:43.404121 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.404130 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.404991 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.404994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.406004 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.406013 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.406707 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.406710 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.406800 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.407840 (+ 1040us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:43.407850 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:43.408254 (+ 404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:43.410009 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.410012 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.410015 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.410252 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.410507 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.410510 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.410512 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.410603 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.411037 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:43.411039 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:43.411117 (+ 78us) client_negotiation.cc:769] Sending connection context
0812 03:03:43.411203 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 03:03:43.411341 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":35,"thread_start_us":102,"threads_started":1}
I20260812 03:03:43.411540 15122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.403184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38088 (local address 127.5.96.190:39179)
0812 03:03:43.403307 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:43.403312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:43.403489 (+ 177us) server_negotiation.cc:407] Connection header received
0812 03:03:43.403590 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:43.403593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:43.403636 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:43.403730 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:43.404260 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.404878 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.406134 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.406471 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.406543 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.408390 (+ 1847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:43.408394 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:43.408397 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:43.408423 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:43.409922 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.410342 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.410346 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.410348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.410426 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.410704 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.410708 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.410710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.410939 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:43.411036 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:43.411302 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 03:03:43.411408 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:03:43.416246 15377 heartbeater.cc:503] Master 127.5.96.190:39179 was elected leader, sending a full tablet report...
I20260812 03:03:43.416453 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 15155
I20260812 03:03:43.440114 16281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.404066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54094 (local address 127.5.96.189:44711)
0812 03:03:43.404200 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:43.404204 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:43.404866 (+ 662us) server_negotiation.cc:407] Connection header received
0812 03:03:43.405030 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:43.405034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:43.405079 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:43.405126 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:43.405705 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.406488 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.408189 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.408513 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.408637 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.409787 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:43.409791 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:43.409794 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:43.409821 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:43.411730 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.413203 (+ 1473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.413206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.413208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.413311 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.438667 (+ 25356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.438672 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.438674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.438955 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:43.439304 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:43.439805 (+ 501us) server_negotiation.cc:299] Negotiation successful
0812 03:03:43.439945 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 03:03:43.440534 16313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.404194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:44711 (local address 127.0.0.1:54094)
0812 03:03:43.404790 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:43.404802 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:43.404931 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:43.405274 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:43.405277 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:43.405460 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:43.405570 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.405578 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.406645 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.406648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.408045 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.408057 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.408772 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.408776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.408864 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.409248 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:43.409262 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:43.409637 (+ 375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:43.412133 (+ 2496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.412136 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.412138 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.412378 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.413401 (+ 1023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.413404 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.413406 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.438268 (+ 24862us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.439346 (+ 1078us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:43.439349 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:43.439447 (+ 98us) client_negotiation.cc:769] Sending connection context
0812 03:03:43.439553 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 03:03:43.440292 (+ 739us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"mutex_wait_us":30,"thread_start_us":85,"threads_started":1}
Aug 12 03:03:43 dist-test-slave-6bbx krb5kdc[14984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503822, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.190@KRBTEST.COM for oryx/127.5.96.188@KRBTEST.COM
I20260812 03:03:43.446105 16312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.403565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:38451 (local address 127.0.0.1:50038)
0812 03:03:43.404490 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:43.404503 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:43.404604 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:43.404890 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:43.404893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:43.405115 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:43.405263 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.405271 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.406171 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.406173 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.406927 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.406936 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.407542 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.407546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.407645 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.439334 (+ 31689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:43.439348 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:43.439792 (+ 444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:43.444154 (+ 4362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.444157 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.444159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.444644 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.444970 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.444973 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.444975 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.445080 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.445561 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:43.445564 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:43.445662 (+ 98us) client_negotiation.cc:769] Sending connection context
0812 03:03:43.445778 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 03:03:43.445906 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":866,"mutex_wait_us":4,"thread_start_us":113,"threads_started":1}
I20260812 03:03:43.446497 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:44821
--local_ip_for_outbound_sockets=127.5.96.129
--tserver_master_addrs=127.5.96.190:39179
--webserver_port=38821
--webserver_interface=127.5.96.129
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.129.keytab
--principal=oryx/127.5.96.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:43163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.96.190:39179,127.5.96.189:44711
--tserver_master_addrs=127.5.96.190:39179,127.5.96.189:44711,127.5.96.188:38451 with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:03:43.446630 16279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.403887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50038 (local address 127.5.96.188:38451)
0812 03:03:43.403987 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:43.403991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:43.404583 (+ 592us) server_negotiation.cc:407] Connection header received
0812 03:03:43.404679 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:43.404682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:43.404729 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:43.404795 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:43.405387 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.406079 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.407040 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.407265 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.407359 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.442739 (+ 35380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:43.442743 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:43.442746 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:43.442770 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:43.444052 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.444763 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.444766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.444768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.444845 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.445196 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.445199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.445200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.445400 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:43.445505 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:43.446401 (+ 896us) server_negotiation.cc:299] Negotiation successful
0812 03:03:43.446507 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:03:43.447045 15999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.414753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:38451 (local address 127.0.0.1:50048)
0812 03:03:43.414859 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:43.414872 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:43.414940 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:43.415249 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:43.415252 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:43.415426 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:43.415515 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.415521 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.416511 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.416515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.417161 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.417170 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.438739 (+ 21569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.438744 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.438852 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.439989 (+ 1137us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:43.440004 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:43.441463 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:43.445244 (+ 3781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.445259 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.445261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.445489 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.445806 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.445809 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.445811 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.445902 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.446548 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:43.446552 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:43.446649 (+ 97us) client_negotiation.cc:769] Sending connection context
0812 03:03:43.446738 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 03:03:43.446895 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 03:03:43.448000 16315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.414818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50048 (local address 127.5.96.188:38451)
0812 03:03:43.415052 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:43.415058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:43.415068 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:03:43.415118 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:43.415120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:43.415156 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:43.415224 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:43.415623 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.416287 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.438229 (+ 21942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.438595 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.439070 (+ 475us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.442269 (+ 3199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:43.442272 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:43.442273 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:43.442294 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:43.445138 (+ 2844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.445596 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.445598 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.445600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.445698 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.446009 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.446012 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.446014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.446437 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:43.447712 (+ 1275us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:43.447793 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 03:03:43.447878 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":84,"threads_started":1}
I20260812 03:03:43.449049 15058 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.96.188:38451: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5137614d58be42dbadab04e8e76d1a54"
member_type: NON_VOTER
last_known_addr {
host: "127.5.96.188"
port: 38451
}
attrs {
promote: true
}
}
dest_uuid: "6d88e20979594dfc8777fd21df6c1785"
cas_config_opid_index: 5
I20260812 03:03:43.449653 15058 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: NON_VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: true } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: NON_VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: true } }
I20260812 03:03:43.450850 16007 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 5137614d58be42dbadab04e8e76d1a54 (127.5.96.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: NON_VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: true } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: NON_VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: true } } }
I20260812 03:03:43.452453 16007 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.96.188:38451
I20260812 03:03:43.452618 16007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6d88e20979594dfc8777fd21df6c1785" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: NON_VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: true } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: NON_VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: true } } }
I20260812 03:03:43.452714 16007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:43.453190 16222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:43.454543 16222 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [term 1 FOLLOWER]: Refusing update from remote peer 6d88e20979594dfc8777fd21df6c1785: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 03:03:43.455276 16011 master.cc:537] Master@127.5.96.188:38451 shutting down...
I20260812 03:03:43.456934 15999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.450705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:44711 (local address 127.0.0.1:54100)
0812 03:03:43.452074 (+ 1369us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:43.452087 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:43.452165 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:43.452612 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:43.452616 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:43.452634 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:43.452718 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.452724 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.453901 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.453905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.455142 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.455154 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.456493 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.456497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.456597 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.456649 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 03:03:43.456726 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 03:03:43.456816 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1296}
I20260812 03:03:43.457356 16281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.450799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54100 (local address 127.5.96.189:44711)
0812 03:03:43.450903 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:43.450907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:43.452244 (+ 1337us) server_negotiation.cc:407] Connection header received
0812 03:03:43.452292 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:43.452295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:43.452354 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:43.452440 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:43.452958 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.453711 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.455344 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.456014 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.456119 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.456282 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:43.457161 (+ 879us) server_negotiation.cc:299] Negotiation successful
0812 03:03:43.457240 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:03:43.457185 16003 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: NON_VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:43.459316 16080 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [term 1 LEARNER]: Refusing update from remote peer 6d88e20979594dfc8777fd21df6c1785: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 03:03:43.459753 16003 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: NON_VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260812 03:03:43.460129 15034 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 -> Peer 5137614d58be42dbadab04e8e76d1a54 (127.5.96.188:38451): Couldn't send request to peer 5137614d58be42dbadab04e8e76d1a54. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 03:03:43.461601 16080 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 5137614d58be42dbadab04e8e76d1a54 (127.5.96.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: NON_VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: true } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: NON_VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: true } } }
I20260812 03:03:43.463900 16134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d88e20979594dfc8777fd21df6c1785. Latest consensus state: current_term: 1 leader_uuid: "6d88e20979594dfc8777fd21df6c1785" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: NON_VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: true } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: NON_VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: true } } }
I20260812 03:03:43.464020 16134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [sys.catalog]: This master's current role is: LEARNER
I20260812 03:03:43.464108 16134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6d88e20979594dfc8777fd21df6c1785" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: NON_VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: true } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: NON_VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: true } } }
I20260812 03:03:43.464178 16134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [sys.catalog]: This master's current role is: LEARNER
I20260812 03:03:43.464870 16003 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785: attempting to promote NON_VOTER 985406e5f9d44f50b81a6191e3026349 to VOTER
I20260812 03:03:43.465286 16003 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: NON_VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: true } }
W20260812 03:03:43.466830 15034 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 -> Peer 5137614d58be42dbadab04e8e76d1a54 (127.5.96.188:38451): Couldn't send request to peer 5137614d58be42dbadab04e8e76d1a54. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 03:03:43.466976 16011 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:43.467197 16079 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [term 1 LEARNER]: Refusing update from remote peer 6d88e20979594dfc8777fd21df6c1785: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 03:03:43.467242 16011 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:43.467336 16011 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54: stopping tablet replica
I20260812 03:03:43.467619 16007 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 03:03:43.469235 16007 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 985406e5f9d44f50b81a6191e3026349 (127.5.96.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: NON_VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: true } } }
I20260812 03:03:43.471349 16080 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 985406e5f9d44f50b81a6191e3026349 (127.5.96.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: NON_VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: true } } }
I20260812 03:03:43.472048 16001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6d88e20979594dfc8777fd21df6c1785" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: NON_VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: true } } }
I20260812 03:03:43.472157 16001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:43.472381 16001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6d88e20979594dfc8777fd21df6c1785" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: NON_VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: true } } }
I20260812 03:03:43.472467 16001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:43.472909 16319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6d88e20979594dfc8777fd21df6c1785" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: NON_VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: true } } }
I20260812 03:03:43.473019 16319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:03:43.481834 16011 master.cc:552] Master@127.5.96.188:38451 shutdown complete.
I20260812 03:03:43.481886 16011 master_runner.cc:305] Clearing existing system tablet
I20260812 03:03:43.482120 16011 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:43.482177 16011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:43.483410 16011 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:03:43.484309 16321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:43.484550 16011 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:43.484830 16011 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:03:43.484921 16011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-2/wal
uuid: "5137614d58be42dbadab04e8e76d1a54"
format_stamp: "Formatted at 2026-08-12 03:03:43 on dist-test-slave-6bbx"
server_key: "83b2c0322f4938cd7b2baf5f088de078"
server_key_iv: "d791079d382ae3b0927430d9ca1b0e71"
server_key_version: "encryptionkey@0"
I20260812 03:03:43.486986 16011 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:43.489527 16011 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 03:03:43.489673 16011 log.cc:1196] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 03:03:43.490000 16011 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54: Deleting consensus metadata
I20260812 03:03:43.490216 16011 master_runner.cc:315] Copying system tablet from 127.5.96.190:39179
I20260812 03:03:43.491348 16011 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54: tablet copy: Beginning tablet copy session from remote peer at address 127.5.96.190:39179
I20260812 03:03:43.500238 16330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.491916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.0.0.1:38098)
0812 03:03:43.492193 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:43.492206 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:43.492319 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:43.492560 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:43.492563 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:43.492790 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:43.492915 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.492923 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.493964 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.493967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.494897 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.494928 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.495467 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.495471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.495569 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.496131 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:43.496146 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:43.496549 (+ 403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:43.498533 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.498536 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.498538 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.498792 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.499070 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.499073 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.499075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.499164 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.499626 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:43.499629 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:43.499803 (+ 174us) client_negotiation.cc:769] Sending connection context
0812 03:03:43.499910 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 03:03:43.500043 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":82,"threads_started":1}
I20260812 03:03:43.500238 15122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.492025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38098 (local address 127.5.96.190:39179)
0812 03:03:43.492134 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:43.492138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:43.492284 (+ 146us) server_negotiation.cc:407] Connection header received
0812 03:03:43.492395 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:43.492398 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:43.492447 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:43.492533 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:43.493031 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.493852 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.495040 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.495357 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.495478 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.496690 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:43.496694 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:43.496697 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:43.496723 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:43.498436 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.498891 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.498894 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.498896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.498983 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.499265 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.499268 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.499270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.499533 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:43.499629 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:43.499982 (+ 353us) server_negotiation.cc:299] Negotiation successful
0812 03:03:43.500094 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:03:43.500964 15086 tablet_copy_service.cc:140] P 6d88e20979594dfc8777fd21df6c1785: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5137614d58be42dbadab04e8e76d1a54 ({username='oryx', principal='oryx/127.5.96.188@KRBTEST.COM'} at 127.0.0.1:38098)
I20260812 03:03:43.501082 15086 tablet_copy_service.cc:161] P 6d88e20979594dfc8777fd21df6c1785: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5137614d58be42dbadab04e8e76d1a54 at {username='oryx', principal='oryx/127.5.96.188@KRBTEST.COM'} at 127.0.0.1:38098: session id = 5137614d58be42dbadab04e8e76d1a54-00000000000000000000000000000000
I20260812 03:03:43.502128 15086 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 03:03:43.503290 16011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 03:03:43.505422 16011 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54: tablet copy: Starting download of 0 data blocks...
I20260812 03:03:43.505657 16011 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54: tablet copy: Starting download of 1 WAL segments...
I20260812 03:03:43.507157 16011 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 03:03:43.508922 15086 tablet_copy_service.cc:342] P 6d88e20979594dfc8777fd21df6c1785: Request end of tablet copy session 5137614d58be42dbadab04e8e76d1a54-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.96.188@KRBTEST.COM'} at 127.0.0.1:38098
I20260812 03:03:43.509001 15086 tablet_copy_service.cc:433] P 6d88e20979594dfc8777fd21df6c1785: ending tablet copy session 5137614d58be42dbadab04e8e76d1a54-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5137614d58be42dbadab04e8e76d1a54
I20260812 03:03:43.516767 16011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:43.518318 16332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:43.518455 16336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:43.520053 16333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:43.520325 16011 server_base.cc:1034] running on GCE node
I20260812 03:03:43.520460 16011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:43.520725 16011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:43.521867 16011 hybrid_clock.cc:648] HybridClock initialized: now 1786503823521854 us; error 30 us; skew 500 ppm
Aug 12 03:03:43 dist-test-slave-6bbx krb5kdc[14984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503823, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:43.523484 16011 init.cc:377] Logged in from keytab as oryx/127.5.96.188@KRBTEST.COM (short username oryx)
I20260812 03:03:43.524087 16011 webserver.cc:460] Webserver started at http://127.5.96.188:40883/ using document root <none> and password file <none>
I20260812 03:03:43.524322 16011 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:43.524415 16011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:43.525736 16011 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:03:43.526669 16346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:43.526909 16011 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:03:43.526981 16011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-2/wal
uuid: "5137614d58be42dbadab04e8e76d1a54"
format_stamp: "Formatted at 2026-08-12 03:03:43 on dist-test-slave-6bbx"
server_key: "83b2c0322f4938cd7b2baf5f088de078"
server_key_iv: "d791079d382ae3b0927430d9ca1b0e71"
server_key_version: "encryptionkey@0"
I20260812 03:03:43.527050 16011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:43.549364 16011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:43.551633 16011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:43.556581 16011 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.188:38451
I20260812 03:03:43.556614 16432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.188:38451 every 8 connection(s)
I20260812 03:03:43.557157 16011 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/master-2/data/info.pb
I20260812 03:03:43.559535 16433 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:03:43.560621 16433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54: Bootstrap starting.
I20260812 03:03:43.571450 16433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:03:43.571751 16433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54: Bootstrap complete.
I20260812 03:03:43.572299 16433 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: NON_VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: true } }
I20260812 03:03:43.572428 16433 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5137614d58be42dbadab04e8e76d1a54, State: Initialized, Role: LEARNER
I20260812 03:03:43.572527 16433 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: NON_VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: true } }
I20260812 03:03:43.572822 16438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: NON_VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: true } } }
I20260812 03:03:43.572937 16433 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:03:43.573027 16438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [sys.catalog]: This master's current role is: LEARNER
I20260812 03:03:43.575829 16458 catalog_manager.cc:1259] Loaded cluster ID: e62bf77d7cc045c1a93036314667f427
I20260812 03:03:43.576085 16458 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54: loading cluster ID for follower catalog manager: success
I20260812 03:03:43.579368 16458 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54: acquiring CA information for follower catalog manager: success
I20260812 03:03:43.580950 16458 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 03:03:43.586628 16316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:43.586834 16316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:43.586874 16316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:43.590704 16316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:43.590749 16316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:43.590816 16316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:03:43.595888 16316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:43163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.129.keytab
--principal=oryx/127.5.96.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:44821
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=38821
--tserver_master_addrs=127.5.96.190:39179,127.5.96.189:44711,127.5.96.188:38451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:43.596738 16316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:43.597615 16316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:43.609472 16470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:43.610502 16467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:43.610718 16468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:43.611542 16316 server_base.cc:1034] running on GCE node
I20260812 03:03:43.612002 16316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:43.612715 16316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:43.613897 16316 hybrid_clock.cc:648] HybridClock initialized: now 1786503823613870 us; error 36 us; skew 500 ppm
Aug 12 03:03:43 dist-test-slave-6bbx krb5kdc[14984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503823, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:43.617095 16316 init.cc:377] Logged in from keytab as oryx/127.5.96.129@KRBTEST.COM (short username oryx)
I20260812 03:03:43.618198 16316 webserver.cc:460] Webserver started at http://127.5.96.129:38821/ using document root <none> and password file <none>
I20260812 03:03:43.618857 16316 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:43.619784 16316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:43.624183 16316 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:03:43.628523 16478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:43.630774 16316 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260812 03:03:43.630906 16316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "ab340b69c0a246b4bfc2a120a824d8cd"
format_stamp: "Formatted at 2026-08-12 03:03:42 on dist-test-slave-6bbx"
server_key: "221a419ffb9b720d7c3e6d0d448d021e"
server_key_iv: "755f866e8f37f0c03e3f53cf6f59f979"
server_key_version: "encryptionkey@0"
I20260812 03:03:43.631494 16316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:43.635192 15608 heartbeater.cc:503] Master 127.5.96.190:39179 was elected leader, sending a full tablet report...
I20260812 03:03:43.654232 16316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:43.656694 16316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:43.656836 16316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:43.657332 16316 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:43.658082 16316 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:43.658128 16316 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:43.658162 16316 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:43.658182 16316 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:43.666250 16316 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:44821
I20260812 03:03:43.666298 16662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:44821 every 8 connection(s)
I20260812 03:03:43.667132 16316 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:03:43.671770 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 16316
I20260812 03:03:43.671968 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 15382
Aug 12 03:03:43 dist-test-slave-6bbx krb5kdc[14984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503823, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.189@KRBTEST.COM
Aug 12 03:03:43 dist-test-slave-6bbx krb5kdc[14984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503823, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM
I20260812 03:03:43.683457 16281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.668839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:35455 (local address 127.5.96.189:44711)
0812 03:03:43.668960 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:43.668964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:43.670485 (+ 1521us) server_negotiation.cc:407] Connection header received
0812 03:03:43.671394 (+ 909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:43.671398 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:43.671451 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:43.671529 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:43.672495 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.673142 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.674368 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.674640 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.674699 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.677669 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:43.677673 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:43.677675 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:43.677698 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:43.679826 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.681738 (+ 1912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.681743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.681746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.681857 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.682403 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.682407 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.682410 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.682603 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:43.682663 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:43.683199 (+ 536us) server_negotiation.cc:299] Negotiation successful
0812 03:03:43.683313 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 03:03:43.684749 16675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.669201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:44711 (local address 127.5.96.129:35455)
0812 03:03:43.670399 (+ 1198us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:43.670408 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:43.670626 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:43.671650 (+ 1024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:43.671653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:43.671840 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:43.672340 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.672358 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.673326 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.673328 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.674241 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.674250 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.674861 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.674864 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.675091 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.676000 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:43.676017 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:43.677505 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:43.681180 (+ 3675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.681186 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.681201 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.681592 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.681999 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.682002 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.682005 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.682264 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.682734 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:43.682740 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:43.682982 (+ 242us) client_negotiation.cc:769] Sending connection context
0812 03:03:43.683582 (+ 600us) client_negotiation.cc:240] Negotiation successful
0812 03:03:43.683893 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1117,"mutex_wait_us":143,"thread_start_us":108,"threads_started":1}
I20260812 03:03:43.686033 16668 heartbeater.cc:347] Connected to a master server at 127.5.96.189:44711
I20260812 03:03:43.686239 16668 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:43.686710 16668 heartbeater.cc:511] Master 127.5.96.189:44711 requested a full tablet report, sending...
I20260812 03:03:43.688336 16054 ts_manager.cc:194] Registered new tserver with Master: ab340b69c0a246b4bfc2a120a824d8cd (127.5.96.129:44821)
I20260812 03:03:43.689386 15122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.668801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:35089 (local address 127.5.96.190:39179)
0812 03:03:43.668948 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:43.668953 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:43.670252 (+ 1299us) server_negotiation.cc:407] Connection header received
0812 03:03:43.671114 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:43.671118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:43.671167 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:43.671256 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:43.676185 (+ 4929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.677168 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.678583 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.678848 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.678931 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.682543 (+ 3612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:43.682549 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:43.682552 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:43.682585 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:43.684885 (+ 2300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.686295 (+ 1410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.686300 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.686302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.686407 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.686891 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.686895 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.686897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.687159 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:43.687244 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:43.689113 (+ 1869us) server_negotiation.cc:299] Negotiation successful
0812 03:03:43.689240 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 03:03:43.689447 16673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.669201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.5.96.129:35089)
0812 03:03:43.670161 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:43.670172 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:43.670626 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:43.671275 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:43.671277 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:43.671624 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:43.676006 (+ 4382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.676018 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.677434 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.677438 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.678458 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.678471 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.678966 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.678971 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.679075 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.679451 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:43.679468 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:43.682376 (+ 2908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:43.685775 (+ 3399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.685779 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.685781 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.686077 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.686586 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.686590 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.686593 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.686703 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.687342 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:43.687346 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:43.688858 (+ 1512us) client_negotiation.cc:769] Sending connection context
0812 03:03:43.689043 (+ 185us) client_negotiation.cc:240] Negotiation successful
0812 03:03:43.689201 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":887,"mutex_wait_us":53,"spinlock_wait_cycles":26880,"thread_start_us":121,"threads_started":1}
I20260812 03:03:43.690160 16667 heartbeater.cc:347] Connected to a master server at 127.5.96.190:39179
I20260812 03:03:43.690249 16667 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:43.690404 16667 heartbeater.cc:511] Master 127.5.96.190:39179 requested a full tablet report, sending...
I20260812 03:03:43.691159 15058 ts_manager.cc:194] Re-registered known tserver with Master: ab340b69c0a246b4bfc2a120a824d8cd (127.5.96.129:44821)
I20260812 03:03:43.691906 15058 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.96.129@KRBTEST.COM'} at 127.5.96.129:35089
Aug 12 03:03:43 dist-test-slave-6bbx krb5kdc[14984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503823, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.188@KRBTEST.COM
I20260812 03:03:43.696022 16672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.669292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:37347 (local address 127.5.96.188:38451)
0812 03:03:43.669526 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:43.669530 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:43.669902 (+ 372us) server_negotiation.cc:407] Connection header received
0812 03:03:43.670831 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:43.670834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:43.670896 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:43.670972 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:43.687912 (+ 16940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.688737 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.690151 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.690633 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.690774 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.693050 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:43.693053 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:43.693056 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:43.693083 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:43.694505 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.694936 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.694938 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.694940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.695008 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.695244 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.695246 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.695247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.695398 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:43.695465 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:43.695784 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 03:03:43.695896 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":67,"threads_started":1}
I20260812 03:03:43.696308 16677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.669201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:38451 (local address 127.5.96.129:37347)
0812 03:03:43.669761 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:43.669811 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:43.670620 (+ 809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:43.671145 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:43.671154 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:43.671622 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:43.687787 (+ 16165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.687794 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.688881 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.688883 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.689972 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.689985 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.690779 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.690782 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.690872 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.691359 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:43.691372 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:43.692889 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:43.694609 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.694612 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.694614 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.694852 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.695080 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.695083 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.695085 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.695172 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.695483 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:43.695486 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:43.695575 (+ 89us) client_negotiation.cc:769] Sending connection context
0812 03:03:43.695725 (+ 150us) client_negotiation.cc:240] Negotiation successful
0812 03:03:43.695884 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":7,"thread_start_us":135,"threads_started":1}
I20260812 03:03:43.696951 16666 heartbeater.cc:347] Connected to a master server at 127.5.96.188:38451
I20260812 03:03:43.697039 16666 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:43.697211 16666 heartbeater.cc:511] Master 127.5.96.188:38451 requested a full tablet report, sending...
I20260812 03:03:43.698195 16378 ts_manager.cc:194] Registered new tserver with Master: ab340b69c0a246b4bfc2a120a824d8cd (127.5.96.129:44821)
I20260812 03:03:43.700556 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:33861
--local_ip_for_outbound_sockets=127.5.96.130
--tserver_master_addrs=127.5.96.190:39179
--webserver_port=43899
--webserver_interface=127.5.96.130
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.130.keytab
--principal=oryx/127.5.96.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:43163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.96.190:39179,127.5.96.189:44711
--tserver_master_addrs=127.5.96.190:39179,127.5.96.189:44711,127.5.96.188:38451 with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:43.808084 16679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:43.808341 16679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:43.808384 16679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:43.812209 16679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:43.812261 16679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:43.812315 16679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:03:43.817104 16679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:43163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.130.keytab
--principal=oryx/127.5.96.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:33861
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=43899
--tserver_master_addrs=127.5.96.190:39179,127.5.96.189:44711,127.5.96.188:38451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:43.817932 16679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:43.818832 16679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:43.827787 16687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:43.827801 16688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:43.827801 16692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:43.827992 16679 server_base.cc:1034] running on GCE node
I20260812 03:03:43.828454 16679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:43.829020 16679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:43.830179 16679 hybrid_clock.cc:648] HybridClock initialized: now 1786503823830159 us; error 33 us; skew 500 ppm
Aug 12 03:03:43 dist-test-slave-6bbx krb5kdc[14984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503823, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:43.832798 16679 init.cc:377] Logged in from keytab as oryx/127.5.96.130@KRBTEST.COM (short username oryx)
I20260812 03:03:43.833581 16679 webserver.cc:460] Webserver started at http://127.5.96.130:43899/ using document root <none> and password file <none>
I20260812 03:03:43.834106 16679 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:43.834205 16679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:43.837910 16679 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:03:43.838856 15831 heartbeater.cc:503] Master 127.5.96.190:39179 was elected leader, sending a full tablet report...
I20260812 03:03:43.839890 16703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:43.841075 16679 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:43.841187 16679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "fbc247aa6b9d4c308db729ea9b3c80a0"
format_stamp: "Formatted at 2026-08-12 03:03:42 on dist-test-slave-6bbx"
server_key: "4adbc5f452ce7b41ccbcfd8606fff969"
server_key_iv: "d1245b68a5a9b358a78d855d24c52e12"
server_key_version: "encryptionkey@0"
I20260812 03:03:43.841650 16679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:43.858870 16679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:43.861862 16679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:43.862004 16679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:43.862517 16679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:43.863300 16679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:43.863348 16679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:43.863384 16679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:43.863405 16679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:43.871594 16679 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:33861
I20260812 03:03:43.871631 16879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:33861 every 8 connection(s)
I20260812 03:03:43.872555 16679 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:03:43.876442 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 16679
I20260812 03:03:43.876602 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 15615
Aug 12 03:03:43 dist-test-slave-6bbx krb5kdc[14984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503823, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM
I20260812 03:03:43.888075 15122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.874268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:55609 (local address 127.5.96.190:39179)
0812 03:03:43.874397 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:43.874401 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:43.876001 (+ 1600us) server_negotiation.cc:407] Connection header received
0812 03:03:43.876649 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:43.876653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:43.876711 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:43.876764 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:43.878904 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.879658 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.881693 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.882032 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.882126 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.884723 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:43.884727 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:43.884729 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:43.884755 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:43.886052 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.886584 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.886587 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.886589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.886667 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.886970 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.886973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.886974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.887166 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:43.887223 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:43.887805 (+ 582us) server_negotiation.cc:299] Negotiation successful
0812 03:03:43.887928 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:03:43.889349 16893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.874740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.5.96.130:55609)
0812 03:03:43.875918 (+ 1178us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:43.875931 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:43.876409 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:43.877154 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:43.877163 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:43.878697 (+ 1534us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:43.878791 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.878796 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.879883 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.879887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.881545 (+ 1658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.881555 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.882309 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.882313 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.882483 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.883035 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:43.883053 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:43.884556 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:43.886184 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.886189 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.886203 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.886479 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.886756 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.886758 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.886759 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.886881 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.887290 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:43.887296 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:43.887546 (+ 250us) client_negotiation.cc:769] Sending connection context
0812 03:03:43.888212 (+ 666us) client_negotiation.cc:240] Negotiation successful
0812 03:03:43.888525 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1077,"mutex_wait_us":156,"thread_start_us":118,"threads_started":1}
Aug 12 03:03:43 dist-test-slave-6bbx krb5kdc[14984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503823, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.189@KRBTEST.COM
I20260812 03:03:43.906606 16886 heartbeater.cc:347] Connected to a master server at 127.5.96.190:39179
I20260812 03:03:43.906903 16886 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:43.907487 16886 heartbeater.cc:511] Master 127.5.96.190:39179 requested a full tablet report, sending...
Aug 12 03:03:43 dist-test-slave-6bbx krb5kdc[14984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503823, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.188@KRBTEST.COM
I20260812 03:03:43.908674 15058 ts_manager.cc:194] Re-registered known tserver with Master: fbc247aa6b9d4c308db729ea9b3c80a0 (127.5.96.130:33861)
I20260812 03:03:43.909415 15058 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.96.130@KRBTEST.COM'} at 127.5.96.130:55609
I20260812 03:03:43.910063 16281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.874269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:60381 (local address 127.5.96.189:44711)
0812 03:03:43.874423 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:43.874428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:43.875644 (+ 1216us) server_negotiation.cc:407] Connection header received
0812 03:03:43.876666 (+ 1022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:43.876669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:43.876717 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:43.876765 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:43.881320 (+ 4555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.882069 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.903282 (+ 21213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.903616 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.903762 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.906052 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:43.906055 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:43.906058 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:43.906087 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:43.907937 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.908530 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.908533 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.908535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.908616 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.909012 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.909016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.909017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.909266 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:43.909344 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:43.909791 (+ 447us) server_negotiation.cc:299] Negotiation successful
0812 03:03:43.909906 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 03:03:43.910987 16895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.874740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:44711 (local address 127.5.96.130:60381)
0812 03:03:43.875430 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:43.875506 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:43.876400 (+ 894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:43.877154 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:43.877168 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:43.877911 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:43.878442 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.878456 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.902568 (+ 24112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.902571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.903144 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.903154 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.903899 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.903903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.903997 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.904495 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:43.904509 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:43.905899 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:43.908140 (+ 2241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.908144 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.908146 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.908430 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.908795 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.908799 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.908801 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.908900 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.909418 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:43.909421 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:43.909578 (+ 157us) client_negotiation.cc:769] Sending connection context
0812 03:03:43.910648 (+ 1070us) client_negotiation.cc:240] Negotiation successful
0812 03:03:43.910799 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":117,"threads_started":1}
I20260812 03:03:43.911803 16672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.874761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:55893 (local address 127.5.96.188:38451)
0812 03:03:43.874892 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:43.874897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:43.875903 (+ 1006us) server_negotiation.cc:407] Connection header received
0812 03:03:43.876643 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:43.876648 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:43.876704 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:43.876771 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:43.903906 (+ 27135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.904576 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.905554 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.905859 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.905958 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.908458 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:43.908462 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:43.908465 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:43.908494 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:43.910043 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.910521 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.910525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.910526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.910603 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:43.910887 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:43.910891 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:43.910892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:43.911102 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:43.911179 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:43.911518 (+ 339us) server_negotiation.cc:299] Negotiation successful
0812 03:03:43.911640 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:03:43.911918 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:43647
--local_ip_for_outbound_sockets=127.5.96.131
--tserver_master_addrs=127.5.96.190:39179
--webserver_port=41877
--webserver_interface=127.5.96.131
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.131.keytab
--principal=oryx/127.5.96.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:43163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.96.190:39179,127.5.96.189:44711
--tserver_master_addrs=127.5.96.190:39179,127.5.96.189:44711,127.5.96.188:38451 with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:03:43.911841 16887 heartbeater.cc:347] Connected to a master server at 127.5.96.189:44711
I20260812 03:03:43.912129 16887 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:43.912312 16887 heartbeater.cc:511] Master 127.5.96.189:44711 requested a full tablet report, sending...
I20260812 03:03:43.912298 16894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.874740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:38451 (local address 127.5.96.130:55893)
0812 03:03:43.875815 (+ 1075us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:43.875828 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:43.876403 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:43.877158 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:43.877164 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:43.903588 (+ 26424us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:43.903753 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.903762 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.904718 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.904732 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.905437 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.905447 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.906094 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.906098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.906183 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.906768 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:43.906783 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:43.908269 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:43.910172 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.910175 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.910177 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.910413 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.910697 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:43.910699 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:43.910701 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:43.910782 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:43.911250 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:43.911252 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:43.911341 (+ 89us) client_negotiation.cc:769] Sending connection context
0812 03:03:43.911988 (+ 647us) client_negotiation.cc:240] Negotiation successful
0812 03:03:43.912156 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":986,"mutex_wait_us":44,"thread_start_us":134,"threads_started":1}
I20260812 03:03:43.913050 16884 heartbeater.cc:347] Connected to a master server at 127.5.96.188:38451
I20260812 03:03:43.913059 16054 ts_manager.cc:194] Registered new tserver with Master: fbc247aa6b9d4c308db729ea9b3c80a0 (127.5.96.130:33861)
I20260812 03:03:43.913136 16884 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:43.913265 16884 heartbeater.cc:511] Master 127.5.96.188:38451 requested a full tablet report, sending...
I20260812 03:03:43.913868 16378 ts_manager.cc:194] Registered new tserver with Master: fbc247aa6b9d4c308db729ea9b3c80a0 (127.5.96.130:33861)
I20260812 03:03:43.987269 16898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.983480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:38451 (local address 127.0.0.1:50060)
0812 03:03:43.983725 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:43.983737 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:43.983860 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:43.984140 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:43.984141 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:43.984155 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:43.984251 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.984257 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.985264 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.985266 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.986026 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:43.986035 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.986933 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.986936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:43.987011 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.987036 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 03:03:43.987082 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 03:03:43.987142 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":65,"threads_started":1}
I20260812 03:03:43.987330 16672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:43.983570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50060 (local address 127.5.96.188:38451)
0812 03:03:43.983745 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:43.983749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:43.983823 (+ 74us) server_negotiation.cc:407] Connection header received
0812 03:03:43.983960 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:43.983963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:43.984030 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:43.984140 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:43.984349 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.985160 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.986147 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:43.986817 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:43.986934 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:43.987087 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:43.987122 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 03:03:43.987190 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 03:03:43.988026 16391 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260812 03:03:43.988409 16438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d88e20979594dfc8777fd21df6c1785. Latest consensus state: current_term: 1 leader_uuid: "6d88e20979594dfc8777fd21df6c1785" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: NON_VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: true } } }
I20260812 03:03:43.988554 16438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [sys.catalog]: This master's current role is: LEARNER
W20260812 03:03:44.021034 16896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:44.021250 16896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:44.021288 16896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:44.025301 16896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:44.025349 16896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:44.025409 16896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:03:44.030357 16896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:43163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.131.keytab
--principal=oryx/127.5.96.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:43647
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=41877
--tserver_master_addrs=127.5.96.190:39179,127.5.96.189:44711,127.5.96.188:38451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:44.031226 16896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:44.032166 16896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:44.041092 16908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:44.041105 16906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:44.041244 16896 server_base.cc:1034] running on GCE node
W20260812 03:03:44.041193 16911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:44.041700 16896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:44.042273 16896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:44.043416 16896 hybrid_clock.cc:648] HybridClock initialized: now 1786503824043392 us; error 35 us; skew 500 ppm
Aug 12 03:03:44 dist-test-slave-6bbx krb5kdc[14984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503824, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:44.046079 16896 init.cc:377] Logged in from keytab as oryx/127.5.96.131@KRBTEST.COM (short username oryx)
I20260812 03:03:44.046867 16896 webserver.cc:460] Webserver started at http://127.5.96.131:41877/ using document root <none> and password file <none>
I20260812 03:03:44.047367 16896 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:44.047449 16896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:44.050917 16896 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:44.052809 16921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:44.053969 16896 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:03:44.054059 16896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "e19a80e5310d4f2396df01f7d289dc05"
format_stamp: "Formatted at 2026-08-12 03:03:42 on dist-test-slave-6bbx"
server_key: "3def5d954e1043456911e6c854335b42"
server_key_iv: "8634de2bce991dc6afa0849ba3b33cd6"
server_key_version: "encryptionkey@0"
I20260812 03:03:44.054526 16896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:44.067909 16896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:44.070223 16896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:44.070354 16896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:44.070806 16896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:44.071544 16896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:44.071588 16896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:44.071622 16896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:44.071642 16896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:44.079802 16896 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:43647
I20260812 03:03:44.079869 17098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:43647 every 8 connection(s)
I20260812 03:03:44.080668 16896 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:03:44.088021 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 16896
Aug 12 03:03:44 dist-test-slave-6bbx krb5kdc[14984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503824, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.189@KRBTEST.COM
Aug 12 03:03:44 dist-test-slave-6bbx krb5kdc[14984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503824, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM
Aug 12 03:03:44 dist-test-slave-6bbx krb5kdc[14984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503824, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.188@KRBTEST.COM
I20260812 03:03:44.096256 16672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:44.082459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:51103 (local address 127.5.96.188:38451)
0812 03:03:44.082667 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:44.082671 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:44.083738 (+ 1067us) server_negotiation.cc:407] Connection header received
0812 03:03:44.084667 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:44.084670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:44.084731 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:44.084805 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:44.085937 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.086747 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.088167 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.088487 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.088554 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:44.092556 (+ 4002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:44.092560 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:44.092563 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:44.092592 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:44.094161 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:44.094685 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:44.094688 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:44.094690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:44.094787 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:44.095085 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:44.095088 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:44.095090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:44.095293 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:44.095366 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:44.095961 (+ 595us) server_negotiation.cc:299] Negotiation successful
0812 03:03:44.096086 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143}
I20260812 03:03:44.096380 15122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:44.082333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:51973 (local address 127.5.96.190:39179)
0812 03:03:44.082582 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:44.082586 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:44.083984 (+ 1398us) server_negotiation.cc:407] Connection header received
0812 03:03:44.084704 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:44.084707 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:44.084774 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:44.084823 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:44.085980 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.086496 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.087999 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.088299 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.088364 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:44.091915 (+ 3551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:44.091919 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:44.091921 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:44.091951 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:44.094043 (+ 2092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:44.094570 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:44.094573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:44.094575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:44.094657 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:44.094966 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:44.094969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:44.094971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:44.095170 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:44.095248 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:44.096132 (+ 884us) server_negotiation.cc:299] Negotiation successful
0812 03:03:44.096237 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170}
I20260812 03:03:44.096833 16281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:44.082445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:46355 (local address 127.5.96.189:44711)
0812 03:03:44.082571 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:44.082575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:44.083923 (+ 1348us) server_negotiation.cc:407] Connection header received
0812 03:03:44.084703 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:44.084706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:44.084763 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:44.084815 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:44.085976 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.086755 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.087903 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.088218 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.088314 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:44.091493 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:44.091496 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:44.091499 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:44.091526 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:44.093852 (+ 2326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:44.094498 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:44.094502 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:44.094503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:44.094593 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:44.094965 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:44.094968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:44.094969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:44.095170 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:44.095266 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:44.096580 (+ 1314us) server_negotiation.cc:299] Negotiation successful
0812 03:03:44.096697 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:03:44.097227 17110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:44.082719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:44711 (local address 127.5.96.131:46355)
0812 03:03:44.083823 (+ 1104us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:44.083836 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:44.084499 (+ 663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:44.084970 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:44.084972 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:44.085396 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:44.085820 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:44.085827 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.086857 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.086860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:44.087761 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:44.087771 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.088511 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.088515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:44.088745 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:44.089470 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:44.089486 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:44.091311 (+ 1825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:44.093983 (+ 2672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:44.093988 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:44.094002 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:44.094389 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:44.094689 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:44.094693 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:44.094695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:44.094861 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:44.095317 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:44.095323 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:44.095683 (+ 360us) client_negotiation.cc:769] Sending connection context
0812 03:03:44.095882 (+ 199us) client_negotiation.cc:240] Negotiation successful
0812 03:03:44.096242 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1018,"mutex_wait_us":167,"spinlock_wait_cycles":32768,"thread_start_us":119,"threads_started":1}
I20260812 03:03:44.098104 17108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:44.082719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.5.96.131:51973)
0812 03:03:44.083911 (+ 1192us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:44.083922 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:44.084499 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:44.084966 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:44.084972 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:44.085371 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:44.085818 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:44.085827 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.086636 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.086639 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:44.087867 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:44.087878 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.088513 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.088517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:44.088751 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:44.089373 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:44.089394 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:44.091754 (+ 2360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:44.094152 (+ 2398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:44.094156 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:44.094158 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:44.094468 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:44.094765 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:44.094768 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:44.094769 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:44.094861 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:44.095317 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:44.095323 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:44.095661 (+ 338us) client_negotiation.cc:769] Sending connection context
0812 03:03:44.096857 (+ 1196us) client_negotiation.cc:240] Negotiation successful
0812 03:03:44.097148 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1104,"mutex_wait_us":305,"spinlock_wait_cycles":19584,"thread_start_us":91,"threads_started":1}
I20260812 03:03:44.098493 17103 heartbeater.cc:347] Connected to a master server at 127.5.96.189:44711
I20260812 03:03:44.098711 17103 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:44.098958 17101 heartbeater.cc:347] Connected to a master server at 127.5.96.190:39179
I20260812 03:03:44.099041 17101 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:44.099260 17101 heartbeater.cc:511] Master 127.5.96.190:39179 requested a full tablet report, sending...
I20260812 03:03:44.099437 17111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:44.082719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:38451 (local address 127.5.96.131:51103)
0812 03:03:44.083591 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:44.083629 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:44.084474 (+ 845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:44.084964 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:44.084970 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:44.085344 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:44.085804 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:44.085818 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.087195 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.087198 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:44.088056 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:44.088064 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.088701 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.088705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:44.088822 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:44.089407 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:44.089419 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:44.092398 (+ 2979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:44.094279 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:44.094282 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:44.094284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:44.094578 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:44.094895 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:44.094897 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:44.094899 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:44.094989 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:44.095423 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:44.095426 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:44.095696 (+ 270us) client_negotiation.cc:769] Sending connection context
0812 03:03:44.095882 (+ 186us) client_negotiation.cc:240] Negotiation successful
0812 03:03:44.096317 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"mutex_wait_us":226,"spinlock_wait_cycles":896,"thread_start_us":134,"threads_started":1}
I20260812 03:03:44.100047 17099 heartbeater.cc:347] Connected to a master server at 127.5.96.188:38451
I20260812 03:03:44.100124 17099 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:44.099259 17103 heartbeater.cc:511] Master 127.5.96.189:44711 requested a full tablet report, sending...
I20260812 03:03:44.100261 17099 heartbeater.cc:511] Master 127.5.96.188:38451 requested a full tablet report, sending...
I20260812 03:03:44.100741 15058 ts_manager.cc:194] Re-registered known tserver with Master: e19a80e5310d4f2396df01f7d289dc05 (127.5.96.131:43647)
I20260812 03:03:44.100835 16054 ts_manager.cc:194] Registered new tserver with Master: e19a80e5310d4f2396df01f7d289dc05 (127.5.96.131:43647)
I20260812 03:03:44.101177 17122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:44.090616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54110 (local address 127.5.96.189:44711)
0812 03:03:44.090846 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:44.090850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:44.091036 (+ 186us) server_negotiation.cc:407] Connection header received
0812 03:03:44.091078 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:44.091081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:44.091129 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:44.091210 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:44.093020 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.093788 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.094944 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.095270 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.095366 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:44.097431 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:44.097435 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:44.097438 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:44.097461 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:44.099071 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:44.099570 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:44.099573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:44.099575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:44.099697 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:44.099999 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:44.100003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:44.100004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:44.100212 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:44.100349 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:44.100913 (+ 564us) server_negotiation.cc:299] Negotiation successful
0812 03:03:44.101028 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":77,"threads_started":1}
I20260812 03:03:44.101394 15058 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.96.131@KRBTEST.COM'} at 127.5.96.131:51973
I20260812 03:03:44.101913 16378 ts_manager.cc:194] Registered new tserver with Master: e19a80e5310d4f2396df01f7d289dc05 (127.5.96.131:43647)
I20260812 03:03:44.107883 17123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:44.091133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50066 (local address 127.5.96.188:38451)
0812 03:03:44.092508 (+ 1375us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:44.092512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:44.093138 (+ 626us) server_negotiation.cc:407] Connection header received
0812 03:03:44.097680 (+ 4542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:44.097684 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:44.097727 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:44.097802 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:44.099534 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.102312 (+ 2778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.103292 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.103639 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.103772 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:44.105010 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:44.105014 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:44.105017 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:44.105043 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:44.106271 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:44.106693 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:44.106697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:44.106699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:44.106798 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:44.107057 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:44.107059 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:44.107060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:44.107203 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:44.107299 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:44.107575 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 03:03:44.107732 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1305,"thread_start_us":88,"threads_started":1}
I20260812 03:03:44.108947 17130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:44.091935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38112 (local address 127.5.96.190:39179)
0812 03:03:44.092640 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:44.092644 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:44.097299 (+ 4655us) server_negotiation.cc:407] Connection header received
0812 03:03:44.097340 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:44.097343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:44.097391 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:44.097461 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:44.102544 (+ 5083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.103028 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.104087 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.104418 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.104515 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:44.105900 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:44.105903 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:44.105905 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:44.105926 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:44.107319 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:44.107798 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:44.107801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:44.107803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:44.107882 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:44.108124 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:44.108128 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:44.108129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:44.108300 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:44.108409 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:44.108653 (+ 244us) server_negotiation.cc:299] Negotiation successful
0812 03:03:44.108791 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":75,"threads_started":1}
I20260812 03:03:44.111161 15058 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38112:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:03:44.114105 15058 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:03:44.116478 16007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6d88e20979594dfc8777fd21df6c1785" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: NON_VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: true } } }
I20260812 03:03:44.116607 16007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:44.119624 16007 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785: attempting to promote NON_VOTER 5137614d58be42dbadab04e8e76d1a54 to VOTER
I20260812 03:03:44.120018 16007 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: false } }
I20260812 03:03:44.121120 16080 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [term 1 FOLLOWER]: Refusing update from remote peer 6d88e20979594dfc8777fd21df6c1785: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 03:03:44.121357 16391 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [term 1 LEARNER]: Refusing update from remote peer 6d88e20979594dfc8777fd21df6c1785: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 03:03:44.121526 17135 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260812 03:03:44.121744 17137 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260812 03:03:44.123147 17137 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 5137614d58be42dbadab04e8e76d1a54 (127.5.96.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: false } } }
I20260812 03:03:44.123593 16391 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 5137614d58be42dbadab04e8e76d1a54 (127.5.96.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: false } } }
I20260812 03:03:44.124799 17137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6d88e20979594dfc8777fd21df6c1785" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: false } } }
I20260812 03:03:44.124894 17137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:44.125011 17137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6d88e20979594dfc8777fd21df6c1785" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: false } } }
I20260812 03:03:44.125092 17137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:44.125252 16438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6d88e20979594dfc8777fd21df6c1785" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: false } } }
I20260812 03:03:44.125357 16438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:03:44.126921 17137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6d88e20979594dfc8777fd21df6c1785" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: false } } }
I20260812 03:03:44.127022 17137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:44.127398 16080 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 5137614d58be42dbadab04e8e76d1a54 (127.5.96.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: false } } }
I20260812 03:03:44.129462 17152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6d88e20979594dfc8777fd21df6c1785" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: false } } }
I20260812 03:03:44.129616 17152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:03:44.136029 16898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:44.129527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:33861 (local address 127.0.0.1:46150)
0812 03:03:44.129659 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:44.129674 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:44.129761 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:44.131101 (+ 1340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:44.131104 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:44.131123 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:44.131220 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:44.131227 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.133466 (+ 2239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.133469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:44.134540 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:44.134550 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.135589 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.135592 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:44.135750 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:44.135772 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:03:44.135836 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 03:03:44.135909 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":19}
I20260812 03:03:44.136099 17156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:44.129809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46150 (local address 127.5.96.130:33861)
0812 03:03:44.130229 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:44.130242 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:44.130268 (+ 26us) server_negotiation.cc:407] Connection header received
0812 03:03:44.130356 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:44.130361 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:44.130939 (+ 578us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:44.131384 (+ 445us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:44.132048 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.133354 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.134671 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.135458 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.135685 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:44.135840 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:44.135889 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 03:03:44.135975 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":98,"threads_started":1}
I20260812 03:03:44.136698 17160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:44.130465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:44821 (local address 127.0.0.1:42204)
0812 03:03:44.131233 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:44.131245 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:44.131336 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:44.131973 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:44.131976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:44.131993 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:44.132073 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:44.132079 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.133610 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.133613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:44.134972 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:44.134984 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.136270 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.136274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:44.136359 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:44.136382 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:03:44.136457 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 03:03:44.136524 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"mutex_wait_us":106,"thread_start_us":100,"threads_started":1}
I20260812 03:03:44.136725 17162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:44.130852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42204 (local address 127.5.96.129:44821)
0812 03:03:44.131548 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:44.131562 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:44.131582 (+ 20us) server_negotiation.cc:407] Connection header received
0812 03:03:44.131652 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:44.131656 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:44.131842 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:44.131974 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:44.132163 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.133488 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.135111 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.135941 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.136034 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:44.136202 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:44.136505 (+ 303us) server_negotiation.cc:299] Negotiation successful
0812 03:03:44.136591 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":60,"threads_started":1}
I20260812 03:03:44.138613 16789 tablet_service.cc:1467] Processing CreateTablet for tablet ae3cb03e0e244de79e2518410cfede74 (DEFAULT_TABLE table=test-table [id=34a40460141a4b8b916702b2e7ecd9e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:44.139526 16789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae3cb03e0e244de79e2518410cfede74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:44.139449 16562 tablet_service.cc:1467] Processing CreateTablet for tablet ae3cb03e0e244de79e2518410cfede74 (DEFAULT_TABLE table=test-table [id=34a40460141a4b8b916702b2e7ecd9e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:44.140478 16562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae3cb03e0e244de79e2518410cfede74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:44.142409 17157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:44.130070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:43647 (local address 127.0.0.1:45448)
0812 03:03:44.130517 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:44.130530 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:44.130619 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:44.134166 (+ 3547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:44.134169 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:44.134185 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:44.134261 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:44.134266 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.135578 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.135581 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:44.140774 (+ 5193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:44.140785 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.141988 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.141992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:44.142078 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:44.142117 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 03:03:44.142186 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 03:03:44.142275 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":23,"thread_start_us":94,"threads_started":1}
I20260812 03:03:44.142895 17159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:44.130454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45448 (local address 127.5.96.131:43647)
0812 03:03:44.133632 (+ 3178us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:44.133642 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:44.133662 (+ 20us) server_negotiation.cc:407] Connection header received
0812 03:03:44.133738 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:44.133743 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:44.133906 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:44.134041 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:44.134373 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.135452 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.140915 (+ 5463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.141697 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.141801 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:44.141926 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:44.142662 (+ 736us) server_negotiation.cc:299] Negotiation successful
0812 03:03:44.142763 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3083,"thread_start_us":88,"threads_started":1}
I20260812 03:03:44.145509 16994 tablet_service.cc:1467] Processing CreateTablet for tablet ae3cb03e0e244de79e2518410cfede74 (DEFAULT_TABLE table=test-table [id=34a40460141a4b8b916702b2e7ecd9e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:44.146514 16994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae3cb03e0e244de79e2518410cfede74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:44.147773 17164 tablet_bootstrap.cc:492] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd: Bootstrap starting.
I20260812 03:03:44.148977 17165 tablet_bootstrap.cc:492] T ae3cb03e0e244de79e2518410cfede74 P fbc247aa6b9d4c308db729ea9b3c80a0: Bootstrap starting.
I20260812 03:03:44.149812 17164 tablet_bootstrap.cc:654] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:44.150483 17164 log.cc:824] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:44.151065 17165 tablet_bootstrap.cc:654] T ae3cb03e0e244de79e2518410cfede74 P fbc247aa6b9d4c308db729ea9b3c80a0: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:44.151912 17165 log.cc:824] T ae3cb03e0e244de79e2518410cfede74 P fbc247aa6b9d4c308db729ea9b3c80a0: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:44.152172 17164 tablet_bootstrap.cc:492] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd: No bootstrap required, opened a new log
I20260812 03:03:44.152491 17164 ts_tablet_manager.cc:1397] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 03:03:44.153815 17165 tablet_bootstrap.cc:492] T ae3cb03e0e244de79e2518410cfede74 P fbc247aa6b9d4c308db729ea9b3c80a0: No bootstrap required, opened a new log
I20260812 03:03:44.153998 17165 ts_tablet_manager.cc:1397] T ae3cb03e0e244de79e2518410cfede74 P fbc247aa6b9d4c308db729ea9b3c80a0: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 03:03:44.153977 17171 tablet_bootstrap.cc:492] T ae3cb03e0e244de79e2518410cfede74 P e19a80e5310d4f2396df01f7d289dc05: Bootstrap starting.
I20260812 03:03:44.156044 17171 tablet_bootstrap.cc:654] T ae3cb03e0e244de79e2518410cfede74 P e19a80e5310d4f2396df01f7d289dc05: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:44.156870 17171 log.cc:824] T ae3cb03e0e244de79e2518410cfede74 P e19a80e5310d4f2396df01f7d289dc05: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:44.156870 17164 raft_consensus.cc:348] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbc247aa6b9d4c308db729ea9b3c80a0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33861 } } peers { permanent_uuid: "ab340b69c0a246b4bfc2a120a824d8cd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 44821 } } peers { permanent_uuid: "e19a80e5310d4f2396df01f7d289dc05" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 43647 } }
I20260812 03:03:44.157174 17164 raft_consensus.cc:374] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:44.157238 17164 raft_consensus.cc:729] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab340b69c0a246b4bfc2a120a824d8cd, State: Initialized, Role: FOLLOWER
I20260812 03:03:44.157312 17165 raft_consensus.cc:348] T ae3cb03e0e244de79e2518410cfede74 P fbc247aa6b9d4c308db729ea9b3c80a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbc247aa6b9d4c308db729ea9b3c80a0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33861 } } peers { permanent_uuid: "ab340b69c0a246b4bfc2a120a824d8cd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 44821 } } peers { permanent_uuid: "e19a80e5310d4f2396df01f7d289dc05" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 43647 } }
I20260812 03:03:44.157513 17165 raft_consensus.cc:374] T ae3cb03e0e244de79e2518410cfede74 P fbc247aa6b9d4c308db729ea9b3c80a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:44.157578 17165 raft_consensus.cc:729] T ae3cb03e0e244de79e2518410cfede74 P fbc247aa6b9d4c308db729ea9b3c80a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbc247aa6b9d4c308db729ea9b3c80a0, State: Initialized, Role: FOLLOWER
I20260812 03:03:44.157670 17164 consensus_queue.cc:260] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbc247aa6b9d4c308db729ea9b3c80a0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33861 } } peers { permanent_uuid: "ab340b69c0a246b4bfc2a120a824d8cd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 44821 } } peers { permanent_uuid: "e19a80e5310d4f2396df01f7d289dc05" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 43647 } }
I20260812 03:03:44.157948 17165 consensus_queue.cc:260] T ae3cb03e0e244de79e2518410cfede74 P fbc247aa6b9d4c308db729ea9b3c80a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbc247aa6b9d4c308db729ea9b3c80a0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33861 } } peers { permanent_uuid: "ab340b69c0a246b4bfc2a120a824d8cd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 44821 } } peers { permanent_uuid: "e19a80e5310d4f2396df01f7d289dc05" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 43647 } }
I20260812 03:03:44.158291 16667 heartbeater.cc:503] Master 127.5.96.190:39179 was elected leader, sending a full tablet report...
I20260812 03:03:44.158685 17164 ts_tablet_manager.cc:1428] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 03:03:44.158835 17165 ts_tablet_manager.cc:1428] T ae3cb03e0e244de79e2518410cfede74 P fbc247aa6b9d4c308db729ea9b3c80a0: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 03:03:44.161576 16886 heartbeater.cc:503] Master 127.5.96.190:39179 was elected leader, sending a full tablet report...
I20260812 03:03:44.164925 17171 tablet_bootstrap.cc:492] T ae3cb03e0e244de79e2518410cfede74 P e19a80e5310d4f2396df01f7d289dc05: No bootstrap required, opened a new log
I20260812 03:03:44.165095 17171 ts_tablet_manager.cc:1397] T ae3cb03e0e244de79e2518410cfede74 P e19a80e5310d4f2396df01f7d289dc05: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20260812 03:03:44.168262 17171 raft_consensus.cc:348] T ae3cb03e0e244de79e2518410cfede74 P e19a80e5310d4f2396df01f7d289dc05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbc247aa6b9d4c308db729ea9b3c80a0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33861 } } peers { permanent_uuid: "ab340b69c0a246b4bfc2a120a824d8cd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 44821 } } peers { permanent_uuid: "e19a80e5310d4f2396df01f7d289dc05" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 43647 } }
W20260812 03:03:44.168467 16669 tablet.cc:2369] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:03:44.168489 17171 raft_consensus.cc:374] T ae3cb03e0e244de79e2518410cfede74 P e19a80e5310d4f2396df01f7d289dc05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:44.168535 17171 raft_consensus.cc:729] T ae3cb03e0e244de79e2518410cfede74 P e19a80e5310d4f2396df01f7d289dc05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19a80e5310d4f2396df01f7d289dc05, State: Initialized, Role: FOLLOWER
I20260812 03:03:44.168926 17171 consensus_queue.cc:260] T ae3cb03e0e244de79e2518410cfede74 P e19a80e5310d4f2396df01f7d289dc05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbc247aa6b9d4c308db729ea9b3c80a0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33861 } } peers { permanent_uuid: "ab340b69c0a246b4bfc2a120a824d8cd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 44821 } } peers { permanent_uuid: "e19a80e5310d4f2396df01f7d289dc05" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 43647 } }
I20260812 03:03:44.169489 17101 heartbeater.cc:503] Master 127.5.96.190:39179 was elected leader, sending a full tablet report...
I20260812 03:03:44.169716 17171 ts_tablet_manager.cc:1428] T ae3cb03e0e244de79e2518410cfede74 P e19a80e5310d4f2396df01f7d289dc05: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 03:03:44.200356 17174 raft_consensus.cc:482] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:03:44.200510 17174 raft_consensus.cc:504] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbc247aa6b9d4c308db729ea9b3c80a0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33861 } } peers { permanent_uuid: "ab340b69c0a246b4bfc2a120a824d8cd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 44821 } } peers { permanent_uuid: "e19a80e5310d4f2396df01f7d289dc05" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 43647 } }
I20260812 03:03:44.201642 17174 leader_election.cc:288] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbc247aa6b9d4c308db729ea9b3c80a0 (127.5.96.130:33861), e19a80e5310d4f2396df01f7d289dc05 (127.5.96.131:43647)
I20260812 03:03:44.205950 17156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:44.201990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:35235 (local address 127.5.96.130:33861)
0812 03:03:44.202123 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:44.202127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:44.202244 (+ 117us) server_negotiation.cc:407] Connection header received
0812 03:03:44.202431 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:44.202434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:44.202478 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:44.202536 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:44.202794 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.203287 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.204720 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.205316 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.205421 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:44.205470 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:44.205760 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 03:03:44.205837 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:03:44.206287 17159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:44.201921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:50199 (local address 127.5.96.131:43647)
0812 03:03:44.202237 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:44.202241 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:44.202253 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:03:44.202364 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:44.202367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:44.202413 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:44.202477 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:44.202738 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.203507 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.204821 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.205350 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.205463 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:44.205513 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:44.206125 (+ 612us) server_negotiation.cc:299] Negotiation successful
0812 03:03:44.206200 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241}
I20260812 03:03:44.205991 17182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:44.201819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:43647 (local address 127.5.96.129:50199)
0812 03:03:44.202165 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:44.202178 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:44.202272 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:44.202500 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:44.202502 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:44.202517 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:44.202630 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:44.202637 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.203609 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.203612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:44.204720 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:44.204729 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.205575 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.205578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:44.205692 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:44.205715 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:03:44.205786 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 03:03:44.205857 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":61,"thread_start_us":71,"threads_started":1}
I20260812 03:03:44.206673 17181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:44.201819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:33861 (local address 127.5.96.129:35235)
0812 03:03:44.202157 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:44.202173 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:44.202313 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:44.202609 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:44.202611 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:44.202626 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:44.202704 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:44.202710 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.203391 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.203395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:44.204597 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:44.204609 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.205527 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.205531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:44.205633 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:44.205655 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:03:44.206071 (+ 416us) client_negotiation.cc:240] Negotiation successful
0812 03:03:44.206143 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":70,"threads_started":1}
I20260812 03:03:44.206945 17021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae3cb03e0e244de79e2518410cfede74" candidate_uuid: "ab340b69c0a246b4bfc2a120a824d8cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e19a80e5310d4f2396df01f7d289dc05" is_pre_election: true
I20260812 03:03:44.207211 17021 raft_consensus.cc:2454] T ae3cb03e0e244de79e2518410cfede74 P e19a80e5310d4f2396df01f7d289dc05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab340b69c0a246b4bfc2a120a824d8cd in term 0.
I20260812 03:03:44.207224 16811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae3cb03e0e244de79e2518410cfede74" candidate_uuid: "ab340b69c0a246b4bfc2a120a824d8cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbc247aa6b9d4c308db729ea9b3c80a0" is_pre_election: true
I20260812 03:03:44.207460 16811 raft_consensus.cc:2454] T ae3cb03e0e244de79e2518410cfede74 P fbc247aa6b9d4c308db729ea9b3c80a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab340b69c0a246b4bfc2a120a824d8cd in term 0.
I20260812 03:03:44.207734 16483 leader_election.cc:302] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab340b69c0a246b4bfc2a120a824d8cd, e19a80e5310d4f2396df01f7d289dc05; no voters:
I20260812 03:03:44.208016 17174 raft_consensus.cc:2781] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:03:44.208099 17174 raft_consensus.cc:482] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:03:44.208142 17174 raft_consensus.cc:3037] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:44.209132 17174 raft_consensus.cc:504] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbc247aa6b9d4c308db729ea9b3c80a0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33861 } } peers { permanent_uuid: "ab340b69c0a246b4bfc2a120a824d8cd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 44821 } } peers { permanent_uuid: "e19a80e5310d4f2396df01f7d289dc05" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 43647 } }
I20260812 03:03:44.209421 17174 leader_election.cc:288] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd [CANDIDATE]: Term 1 election: Requested vote from peers fbc247aa6b9d4c308db729ea9b3c80a0 (127.5.96.130:33861), e19a80e5310d4f2396df01f7d289dc05 (127.5.96.131:43647)
I20260812 03:03:44.209702 16811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae3cb03e0e244de79e2518410cfede74" candidate_uuid: "ab340b69c0a246b4bfc2a120a824d8cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbc247aa6b9d4c308db729ea9b3c80a0"
I20260812 03:03:44.209766 17021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae3cb03e0e244de79e2518410cfede74" candidate_uuid: "ab340b69c0a246b4bfc2a120a824d8cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e19a80e5310d4f2396df01f7d289dc05"
I20260812 03:03:44.209810 16811 raft_consensus.cc:3037] T ae3cb03e0e244de79e2518410cfede74 P fbc247aa6b9d4c308db729ea9b3c80a0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:44.209869 17021 raft_consensus.cc:3037] T ae3cb03e0e244de79e2518410cfede74 P e19a80e5310d4f2396df01f7d289dc05 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:44.210764 16811 raft_consensus.cc:2454] T ae3cb03e0e244de79e2518410cfede74 P fbc247aa6b9d4c308db729ea9b3c80a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab340b69c0a246b4bfc2a120a824d8cd in term 1.
I20260812 03:03:44.210780 17021 raft_consensus.cc:2454] T ae3cb03e0e244de79e2518410cfede74 P e19a80e5310d4f2396df01f7d289dc05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab340b69c0a246b4bfc2a120a824d8cd in term 1.
I20260812 03:03:44.211045 16485 leader_election.cc:302] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab340b69c0a246b4bfc2a120a824d8cd, fbc247aa6b9d4c308db729ea9b3c80a0; no voters:
I20260812 03:03:44.211232 17174 raft_consensus.cc:2781] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:44.211390 17174 raft_consensus.cc:686] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd [term 1 LEADER]: Becoming Leader. State: Replica: ab340b69c0a246b4bfc2a120a824d8cd, State: Running, Role: LEADER
I20260812 03:03:44.211699 17174 consensus_queue.cc:237] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbc247aa6b9d4c308db729ea9b3c80a0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33861 } } peers { permanent_uuid: "ab340b69c0a246b4bfc2a120a824d8cd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 44821 } } peers { permanent_uuid: "e19a80e5310d4f2396df01f7d289dc05" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 43647 } }
I20260812 03:03:44.214715 15057 catalog_manager.cc:5495] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd reported cstate change: term changed from 0 to 1, leader changed from <none> to ab340b69c0a246b4bfc2a120a824d8cd (127.5.96.129). New cstate: current_term: 1 leader_uuid: "ab340b69c0a246b4bfc2a120a824d8cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbc247aa6b9d4c308db729ea9b3c80a0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab340b69c0a246b4bfc2a120a824d8cd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 44821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e19a80e5310d4f2396df01f7d289dc05" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 43647 } health_report { overall_health: UNKNOWN } } }
I20260812 03:03:44.231487 17162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:44.228750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42220 (local address 127.5.96.129:44821)
0812 03:03:44.228848 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:44.228851 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:44.228863 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:03:44.228986 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:44.228988 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:44.229034 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:44.229112 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:03:44.229294 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.229745 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.230459 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.230643 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.230703 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:44.230906 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:03:44.231257 (+ 351us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:03:44.231338 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:44.231366 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 03:03:44.231404 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 03:03:44.236475 17021 raft_consensus.cc:1261] T ae3cb03e0e244de79e2518410cfede74 P e19a80e5310d4f2396df01f7d289dc05 [term 1 FOLLOWER]: Refusing update from remote peer ab340b69c0a246b4bfc2a120a824d8cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:44.236479 16811 raft_consensus.cc:1261] T ae3cb03e0e244de79e2518410cfede74 P fbc247aa6b9d4c308db729ea9b3c80a0 [term 1 FOLLOWER]: Refusing update from remote peer ab340b69c0a246b4bfc2a120a824d8cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:44.237115 17183 consensus_queue.cc:1035] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbc247aa6b9d4c308db729ea9b3c80a0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:44.237278 17174 consensus_queue.cc:1035] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e19a80e5310d4f2396df01f7d289dc05" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 43647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:44.244608 17186 mvcc.cc:205] Tried to move back new op lower bound from 7317519664066797568 to 7317519663973277696. Current Snapshot: MvccSnapshot[applied={T|T < 7317519664066797568}]
I20260812 03:03:44.247898 17187 mvcc.cc:205] Tried to move back new op lower bound from 7317519664066797568 to 7317519663973277696. Current Snapshot: MvccSnapshot[applied={T|T < 7317519664066797568}]
I20260812 03:03:44.248291 17188 mvcc.cc:205] Tried to move back new op lower bound from 7317519664066797568 to 7317519663973277696. Current Snapshot: MvccSnapshot[applied={T|T < 7317519664066797568}]
I20260812 03:03:44.253075 15058 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:44.253203 15058 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:44.256938 15058 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6d88e20979594dfc8777fd21df6c1785: Sending DeleteTablet for 3 replicas of tablet ae3cb03e0e244de79e2518410cfede74
I20260812 03:03:44.257861 16994 tablet_service.cc:1514] Processing DeleteTablet for tablet ae3cb03e0e244de79e2518410cfede74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:44 UTC) from {username='oryx', principal='oryx/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:45448
I20260812 03:03:44.257907 16562 tablet_service.cc:1514] Processing DeleteTablet for tablet ae3cb03e0e244de79e2518410cfede74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:44 UTC) from {username='oryx', principal='oryx/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:42204
I20260812 03:03:44.257958 16789 tablet_service.cc:1514] Processing DeleteTablet for tablet ae3cb03e0e244de79e2518410cfede74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:44 UTC) from {username='oryx', principal='oryx/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:46150
I20260812 03:03:44.258591 17206 tablet_replica.cc:330] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd: stopping tablet replica
I20260812 03:03:44.258826 17207 tablet_replica.cc:330] T ae3cb03e0e244de79e2518410cfede74 P fbc247aa6b9d4c308db729ea9b3c80a0: stopping tablet replica
I20260812 03:03:44.258890 17206 raft_consensus.cc:2229] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:03:44.259112 17207 raft_consensus.cc:2229] T ae3cb03e0e244de79e2518410cfede74 P fbc247aa6b9d4c308db729ea9b3c80a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:44.259193 17206 raft_consensus.cc:2258] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:44.259505 17204 tablet_replica.cc:330] T ae3cb03e0e244de79e2518410cfede74 P e19a80e5310d4f2396df01f7d289dc05: stopping tablet replica
I20260812 03:03:44.259794 17204 raft_consensus.cc:2229] T ae3cb03e0e244de79e2518410cfede74 P e19a80e5310d4f2396df01f7d289dc05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:44.260057 17204 raft_consensus.cc:2258] T ae3cb03e0e244de79e2518410cfede74 P e19a80e5310d4f2396df01f7d289dc05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:44.260715 17206 ts_tablet_manager.cc:1905] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:44.261384 17204 ts_tablet_manager.cc:1905] T ae3cb03e0e244de79e2518410cfede74 P e19a80e5310d4f2396df01f7d289dc05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:44.261667 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 15004
I20260812 03:03:44.274521 17207 raft_consensus.cc:2258] T ae3cb03e0e244de79e2518410cfede74 P fbc247aa6b9d4c308db729ea9b3c80a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:44.275563 17204 ts_tablet_manager.cc:1918] T ae3cb03e0e244de79e2518410cfede74 P e19a80e5310d4f2396df01f7d289dc05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:03:44.275647 17204 log.cc:1196] T ae3cb03e0e244de79e2518410cfede74 P e19a80e5310d4f2396df01f7d289dc05: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/wal/wals/ae3cb03e0e244de79e2518410cfede74
I20260812 03:03:44.275926 17204 ts_tablet_manager.cc:1939] T ae3cb03e0e244de79e2518410cfede74 P e19a80e5310d4f2396df01f7d289dc05: Deleting consensus metadata
I20260812 03:03:44.277616 17207 ts_tablet_manager.cc:1905] T ae3cb03e0e244de79e2518410cfede74 P fbc247aa6b9d4c308db729ea9b3c80a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:44.279536 17207 ts_tablet_manager.cc:1918] T ae3cb03e0e244de79e2518410cfede74 P fbc247aa6b9d4c308db729ea9b3c80a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:03:44.279623 17207 log.cc:1196] T ae3cb03e0e244de79e2518410cfede74 P fbc247aa6b9d4c308db729ea9b3c80a0: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/wal/wals/ae3cb03e0e244de79e2518410cfede74
I20260812 03:03:44.279973 17207 ts_tablet_manager.cc:1939] T ae3cb03e0e244de79e2518410cfede74 P fbc247aa6b9d4c308db729ea9b3c80a0: Deleting consensus metadata
I20260812 03:03:44.280927 17206 ts_tablet_manager.cc:1918] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:03:44.281075 17206 log.cc:1196] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-0/wal/wals/ae3cb03e0e244de79e2518410cfede74
I20260812 03:03:44.281405 17206 ts_tablet_manager.cc:1939] T ae3cb03e0e244de79e2518410cfede74 P ab340b69c0a246b4bfc2a120a824d8cd: Deleting consensus metadata
W20260812 03:03:44.285893 16486 connection.cc:535] server connection from 127.0.0.1:42204 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 03:03:44.285920 16930 connection.cc:535] server connection from 127.0.0.1:45448 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 03:03:44.286290 16714 connection.cc:535] server connection from 127.0.0.1:46150 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 03:03:44.291206 17122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:44.259954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54114 (local address 127.5.96.189:44711)
0812 03:03:44.260184 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:44.260189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:44.282466 (+ 22277us) server_negotiation.cc:407] Connection header received
0812 03:03:44.282579 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:44.282582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:44.282632 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:44.282698 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:44.283428 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.283994 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.284776 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.285013 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.285103 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:44.286741 (+ 1638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:44.286744 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:44.286746 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:44.286771 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:44.289419 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:44.289999 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:44.290003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:44.290005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:44.290085 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:44.290381 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:44.290385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:44.290386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:44.290610 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:44.290732 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:44.290941 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 03:03:44.291051 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156}
I20260812 03:03:44.299007 17123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:44.260054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50074 (local address 127.5.96.188:38451)
0812 03:03:44.260268 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:44.260273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:44.291405 (+ 31132us) server_negotiation.cc:407] Connection header received
0812 03:03:44.291458 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:44.291461 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:44.291512 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:44.291580 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:44.292357 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.292859 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.293646 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.293992 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.294093 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:44.295390 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:44.295394 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:44.295397 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:44.295421 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:44.296943 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:44.297390 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:44.297393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:44.297395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:44.297481 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:44.297776 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:44.297780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:44.297781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:44.297982 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:44.298109 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:44.298648 (+ 539us) server_negotiation.cc:299] Negotiation successful
0812 03:03:44.298849 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":153}
I20260812 03:03:44.299089 16281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:44.290411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54120 (local address 127.5.96.189:44711)
0812 03:03:44.290515 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:44.290519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:44.290536 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:03:44.290655 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:44.290658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:44.290702 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:44.290779 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:44.291637 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.292476 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.293677 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.294006 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.294093 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:44.295850 (+ 1757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:44.295854 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:44.295856 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:44.295881 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:44.297392 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:44.297814 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:44.297817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:44.297819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:44.297906 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:44.298238 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:44.298241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:44.298243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:44.298448 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:44.298562 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:44.298849 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 03:03:44.298950 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 03:03:44.306408 16672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:44.297777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50078 (local address 127.5.96.188:38451)
0812 03:03:44.297886 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:44.297890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:44.298411 (+ 521us) server_negotiation.cc:407] Connection header received
0812 03:03:44.298567 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:44.298570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:44.298611 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:44.298656 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:44.300757 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.301206 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.302285 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:44.302474 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:44.302535 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:44.303927 (+ 1392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:44.303930 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:44.303931 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:44.303951 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:44.305115 (+ 1164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:44.305513 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:44.305515 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:44.305516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:44.305576 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:44.305798 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:44.305799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:44.305800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:44.305930 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:44.306015 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:44.306220 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 03:03:44.306299 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":48}
I20260812 03:03:45.248679 17222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.248133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.5.96.129:33001)
0812 03:03:45.248470 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.248565 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":80,"threads_started":1}
I20260812 03:03:45.248697 17223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.248313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.5.96.131:39225)
0812 03:03:45.248525 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.248596 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":63,"threads_started":1}
W20260812 03:03:45.249027 16667 heartbeater.cc:644] Failed to heartbeat to 127.5.96.190:39179 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
W20260812 03:03:45.249027 17101 heartbeater.cc:644] Failed to heartbeat to 127.5.96.190:39179 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
I20260812 03:03:45.249290 17225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.248511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.5.96.130:39947)
0812 03:03:45.248745 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.249201 (+ 456us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":73,"threads_started":1}
W20260812 03:03:45.249552 16886 heartbeater.cc:644] Failed to heartbeat to 127.5.96.190:39179 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
I20260812 03:03:45.757176 17226 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6d88e20979594dfc8777fd21df6c1785)
I20260812 03:03:45.757383 17226 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: false } }
I20260812 03:03:45.757792 17233 raft_consensus.cc:482] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6d88e20979594dfc8777fd21df6c1785)
I20260812 03:03:45.758006 17233 raft_consensus.cc:504] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: false } }
I20260812 03:03:45.758535 17226 leader_election.cc:288] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6d88e20979594dfc8777fd21df6c1785 (127.5.96.190:39179), 985406e5f9d44f50b81a6191e3026349 (127.5.96.189:44711)
I20260812 03:03:45.759361 17235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.758776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.0.0.1:38612)
0812 03:03:45.759174 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.759257 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":96,"threads_started":1}
I20260812 03:03:45.759788 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.759135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.0.0.1:38618)
0812 03:03:45.759589 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.759684 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":71,"threads_started":1}
W20260812 03:03:45.759951 16347 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
W20260812 03:03:45.759986 16044 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
I20260812 03:03:45.760339 17233 leader_election.cc:288] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6d88e20979594dfc8777fd21df6c1785 (127.5.96.190:39179), 5137614d58be42dbadab04e8e76d1a54 (127.5.96.188:38451)
I20260812 03:03:45.761106 17244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.760748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.0.0.1:38622)
0812 03:03:45.760977 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.761033 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
I20260812 03:03:45.761139 17235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.760949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.0.0.1:38628)
0812 03:03:45.761037 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.761077 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 03:03:45.761355 16044 leader_election.cc:334] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6d88e20979594dfc8777fd21df6c1785 (127.5.96.190:39179): Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
W20260812 03:03:45.761633 16347 leader_election.cc:334] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6d88e20979594dfc8777fd21df6c1785 (127.5.96.190:39179): Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
I20260812 03:03:45.763412 17238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.758733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:44711 (local address 127.0.0.1:54128)
0812 03:03:45.759124 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.759138 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:45.759291 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:45.760172 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:45.760175 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:45.760195 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:45.760300 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:45.760307 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.761334 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.761337 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:45.762318 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:45.762328 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.762997 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.763001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:45.763102 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:45.763129 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 03:03:45.763201 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 03:03:45.763282 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":66,"threads_started":1}
I20260812 03:03:45.763427 17242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.758796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54128 (local address 127.5.96.189:44711)
0812 03:03:45.759069 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:45.759074 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:45.759202 (+ 128us) server_negotiation.cc:407] Connection header received
0812 03:03:45.759520 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:45.759524 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:45.759576 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:45.759640 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:45.760433 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.761192 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.762435 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.762890 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.762992 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:45.763053 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:45.763229 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 03:03:45.763306 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":68,"threads_started":1}
I20260812 03:03:45.763957 16080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5137614d58be42dbadab04e8e76d1a54" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "985406e5f9d44f50b81a6191e3026349" is_pre_election: true
I20260812 03:03:45.764150 16080 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5137614d58be42dbadab04e8e76d1a54 in term 1.
I20260812 03:03:45.764485 16354 leader_election.cc:302] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5137614d58be42dbadab04e8e76d1a54, 985406e5f9d44f50b81a6191e3026349; no voters: 6d88e20979594dfc8777fd21df6c1785
I20260812 03:03:45.764678 17226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 03:03:45.764730 17226 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6d88e20979594dfc8777fd21df6c1785)
I20260812 03:03:45.764767 17226 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:03:45.765240 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.759569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:38451 (local address 127.0.0.1:50094)
0812 03:03:45.759933 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.759946 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:45.760030 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:45.761636 (+ 1606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:45.761639 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:45.761658 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:45.761751 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:45.761757 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.762786 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.762789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:45.763907 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:45.763916 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.764871 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.764875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:45.764965 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:45.764989 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:03:45.765066 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 03:03:45.765131 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":85,"threads_started":1}
I20260812 03:03:45.765282 17256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.760398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50094 (local address 127.5.96.188:38451)
0812 03:03:45.761330 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:45.761339 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:45.761355 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:03:45.761401 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:45.761404 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:45.761446 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:45.761532 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:45.761870 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.762677 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.764018 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.764649 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.764753 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:45.764811 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:45.765087 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 03:03:45.765154 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":809,"thread_start_us":84,"threads_started":1}
I20260812 03:03:45.765838 16391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "985406e5f9d44f50b81a6191e3026349" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5137614d58be42dbadab04e8e76d1a54" is_pre_election: true
I20260812 03:03:45.766060 17226 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: false } }
I20260812 03:03:45.766281 16391 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 985406e5f9d44f50b81a6191e3026349 in current term 2: Already voted for candidate 5137614d58be42dbadab04e8e76d1a54 in this term.
I20260812 03:03:45.766422 17226 leader_election.cc:288] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [CANDIDATE]: Term 2 election: Requested vote from peers 6d88e20979594dfc8777fd21df6c1785 (127.5.96.190:39179), 985406e5f9d44f50b81a6191e3026349 (127.5.96.189:44711)
I20260812 03:03:45.766727 16080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5137614d58be42dbadab04e8e76d1a54" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "985406e5f9d44f50b81a6191e3026349"
I20260812 03:03:45.766837 16080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:03:45.766815 16047 leader_election.cc:302] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 985406e5f9d44f50b81a6191e3026349; no voters: 5137614d58be42dbadab04e8e76d1a54, 6d88e20979594dfc8777fd21df6c1785
I20260812 03:03:45.766897 17238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.766583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.0.0.1:38642)
0812 03:03:45.766789 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.766842 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260812 03:03:45.768025 16080 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5137614d58be42dbadab04e8e76d1a54 in term 2.
I20260812 03:03:45.768157 17238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.767961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.0.0.1:38644)
0812 03:03:45.768053 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.768100 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 03:03:45.768143 17233 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 03:03:45.768263 16347 leader_election.cc:334] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6d88e20979594dfc8777fd21df6c1785 (127.5.96.190:39179): Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
I20260812 03:03:45.768368 16354 leader_election.cc:302] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5137614d58be42dbadab04e8e76d1a54, 985406e5f9d44f50b81a6191e3026349; no voters: 6d88e20979594dfc8777fd21df6c1785
I20260812 03:03:45.768532 17226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:03:45.768704 17226 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [term 2 LEADER]: Becoming Leader. State: Replica: 5137614d58be42dbadab04e8e76d1a54, State: Running, Role: LEADER
I20260812 03:03:45.768929 17226 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: false } }
I20260812 03:03:45.770049 17258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5137614d58be42dbadab04e8e76d1a54. Latest consensus state: current_term: 2 leader_uuid: "5137614d58be42dbadab04e8e76d1a54" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: false } } }
I20260812 03:03:45.770143 17258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:45.770435 17263 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:03:45.773425 17263 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:03:45.773888 17263 catalog_manager.cc:1259] Loaded cluster ID: e62bf77d7cc045c1a93036314667f427
I20260812 03:03:45.773942 17263 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:03:45.774611 17263 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:03:45.775094 17263 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54: Loaded TSK: 0
I20260812 03:03:45.775962 17263 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:03:45.802024 16376 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50078:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:03:45.806346 16376 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:03:45.809652 16080 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [term 2 FOLLOWER]: Refusing update from remote peer 5137614d58be42dbadab04e8e76d1a54: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260812 03:03:45.809724 17238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.809483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.0.0.1:38658)
0812 03:03:45.809595 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.809651 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 03:03:45.810163 17258 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260812 03:03:45.810729 17238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.810406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.0.0.1:38664)
0812 03:03:45.810627 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.810670 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 03:03:45.810822 16347 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 -> Peer 6d88e20979594dfc8777fd21df6c1785 (127.5.96.190:39179): Couldn't send request to peer 6d88e20979594dfc8777fd21df6c1785. Status: Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 03:03:45.811628 17233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5137614d58be42dbadab04e8e76d1a54. Latest consensus state: current_term: 2 leader_uuid: "5137614d58be42dbadab04e8e76d1a54" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: false } } }
I20260812 03:03:45.811767 17233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:03:45.812891 17226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "5137614d58be42dbadab04e8e76d1a54" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: false } } }
I20260812 03:03:45.813016 17226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:45.813232 17233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "5137614d58be42dbadab04e8e76d1a54" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d88e20979594dfc8777fd21df6c1785" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39179 } } peers { permanent_uuid: "985406e5f9d44f50b81a6191e3026349" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 44711 } attrs { promote: false } } peers { permanent_uuid: "5137614d58be42dbadab04e8e76d1a54" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 38451 } attrs { promote: false } } }
I20260812 03:03:45.813341 17233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 985406e5f9d44f50b81a6191e3026349 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:03:45.813462 17265 mvcc.cc:205] Tried to move back new op lower bound from 7317519670512160768 to 7317519670351503360. Current Snapshot: MvccSnapshot[applied={T|T < 7317519670512160768}]
I20260812 03:03:45.823915 17238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.819282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:43647 (local address 127.0.0.1:45452)
0812 03:03:45.819791 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.819804 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:45.819879 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:45.820968 (+ 1089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:45.820971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:45.821004 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:45.821128 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:45.821133 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.821877 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.821880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:45.822687 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:45.822697 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.823484 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.823488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:45.823583 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:45.823606 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:03:45.823705 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 03:03:45.823784 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":34,"spinlock_wait_cycles":12032}
I20260812 03:03:45.823916 17270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.819864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45452 (local address 127.5.96.131:43647)
0812 03:03:45.820753 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:45.820757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:45.820767 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:03:45.820816 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:45.820819 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:45.820864 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:45.820933 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:45.821228 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.821777 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.822815 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.823376 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.823471 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:45.823536 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:45.823714 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 03:03:45.823786 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":804,"thread_start_us":90,"threads_started":1}
I20260812 03:03:45.824586 16994 tablet_service.cc:1467] Processing CreateTablet for tablet a7fe3e184a50446d9f08adec090d5628 (DEFAULT_TABLE table=test-table [id=ecc47971ba744b80b5fa9d193d8f55dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:45.824868 16994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7fe3e184a50446d9f08adec090d5628. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:45.825189 17272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.820114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46166 (local address 127.5.96.130:33861)
0812 03:03:45.820342 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:45.820346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:45.820608 (+ 262us) server_negotiation.cc:407] Connection header received
0812 03:03:45.820700 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:45.820703 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:45.820756 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:45.820838 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:45.821284 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.822110 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.823684 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.824444 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.824534 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:45.824618 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:45.824960 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 03:03:45.825044 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":63,"threads_started":1}
I20260812 03:03:45.825640 17273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.820299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:33861 (local address 127.0.0.1:46166)
0812 03:03:45.820505 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.820516 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:45.820609 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:45.820966 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:45.820969 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:45.821000 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:45.821141 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:45.821148 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.822260 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.822263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:45.823491 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:45.823504 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.824721 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.824725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:45.824822 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:45.824844 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:03:45.825388 (+ 544us) client_negotiation.cc:240] Negotiation successful
0812 03:03:45.825473 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":4,"spinlock_wait_cycles":2304,"thread_start_us":83,"threads_started":1}
I20260812 03:03:45.827122 17275 tablet_bootstrap.cc:492] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05: Bootstrap starting.
I20260812 03:03:45.827522 16789 tablet_service.cc:1467] Processing CreateTablet for tablet a7fe3e184a50446d9f08adec090d5628 (DEFAULT_TABLE table=test-table [id=ecc47971ba744b80b5fa9d193d8f55dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:45.827809 16789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7fe3e184a50446d9f08adec090d5628. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:45.828192 17275 tablet_bootstrap.cc:654] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:45.829519 17275 tablet_bootstrap.cc:492] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05: No bootstrap required, opened a new log
I20260812 03:03:45.829629 17275 ts_tablet_manager.cc:1397] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:03:45.829980 17277 tablet_bootstrap.cc:492] T a7fe3e184a50446d9f08adec090d5628 P fbc247aa6b9d4c308db729ea9b3c80a0: Bootstrap starting.
I20260812 03:03:45.830081 17275 raft_consensus.cc:348] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19a80e5310d4f2396df01f7d289dc05" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 43647 } } peers { permanent_uuid: "ab340b69c0a246b4bfc2a120a824d8cd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 44821 } } peers { permanent_uuid: "fbc247aa6b9d4c308db729ea9b3c80a0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33861 } }
I20260812 03:03:45.830210 17275 raft_consensus.cc:374] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:45.830247 17275 raft_consensus.cc:729] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19a80e5310d4f2396df01f7d289dc05, State: Initialized, Role: FOLLOWER
I20260812 03:03:45.830397 17275 consensus_queue.cc:260] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19a80e5310d4f2396df01f7d289dc05" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 43647 } } peers { permanent_uuid: "ab340b69c0a246b4bfc2a120a824d8cd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 44821 } } peers { permanent_uuid: "fbc247aa6b9d4c308db729ea9b3c80a0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33861 } }
I20260812 03:03:45.830734 17275 ts_tablet_manager.cc:1428] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:03:45.831142 17277 tablet_bootstrap.cc:654] T a7fe3e184a50446d9f08adec090d5628 P fbc247aa6b9d4c308db729ea9b3c80a0: Neither blocks nor log segments found. Creating new log.
W20260812 03:03:45.833243 17106 tablet.cc:2369] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:03:45.833549 17280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.831471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.5.96.131:44403)
0812 03:03:45.833397 (+ 1926us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.833454 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1863,"thread_start_us":91,"threads_started":1}
I20260812 03:03:45.834731 17277 tablet_bootstrap.cc:492] T a7fe3e184a50446d9f08adec090d5628 P fbc247aa6b9d4c308db729ea9b3c80a0: No bootstrap required, opened a new log
I20260812 03:03:45.834820 17277 ts_tablet_manager.cc:1397] T a7fe3e184a50446d9f08adec090d5628 P fbc247aa6b9d4c308db729ea9b3c80a0: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260812 03:03:45.835157 17277 raft_consensus.cc:348] T a7fe3e184a50446d9f08adec090d5628 P fbc247aa6b9d4c308db729ea9b3c80a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19a80e5310d4f2396df01f7d289dc05" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 43647 } } peers { permanent_uuid: "ab340b69c0a246b4bfc2a120a824d8cd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 44821 } } peers { permanent_uuid: "fbc247aa6b9d4c308db729ea9b3c80a0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33861 } }
I20260812 03:03:45.835246 17277 raft_consensus.cc:374] T a7fe3e184a50446d9f08adec090d5628 P fbc247aa6b9d4c308db729ea9b3c80a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:45.835275 17277 raft_consensus.cc:729] T a7fe3e184a50446d9f08adec090d5628 P fbc247aa6b9d4c308db729ea9b3c80a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbc247aa6b9d4c308db729ea9b3c80a0, State: Initialized, Role: FOLLOWER
I20260812 03:03:45.835518 17277 consensus_queue.cc:260] T a7fe3e184a50446d9f08adec090d5628 P fbc247aa6b9d4c308db729ea9b3c80a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19a80e5310d4f2396df01f7d289dc05" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 43647 } } peers { permanent_uuid: "ab340b69c0a246b4bfc2a120a824d8cd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 44821 } } peers { permanent_uuid: "fbc247aa6b9d4c308db729ea9b3c80a0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33861 } }
I20260812 03:03:45.835835 17277 ts_tablet_manager.cc:1428] T a7fe3e184a50446d9f08adec090d5628 P fbc247aa6b9d4c308db729ea9b3c80a0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:03:45.835843 17099 heartbeater.cc:503] Master 127.5.96.188:38451 was elected leader, sending a full tablet report...
I20260812 03:03:45.837109 17280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.836330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.5.96.131:44125)
0812 03:03:45.836932 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.836984 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":534}
I20260812 03:03:45.837982 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.837591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.5.96.130:34937)
0812 03:03:45.837857 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.837908 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":88,"threads_started":1}
I20260812 03:03:45.838528 16884 heartbeater.cc:503] Master 127.5.96.188:38451 was elected leader, sending a full tablet report...
I20260812 03:03:45.839391 17235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.819660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:44821 (local address 127.0.0.1:42226)
0812 03:03:45.820004 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.820016 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:45.820084 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:45.828096 (+ 8012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:45.828099 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:45.828119 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:45.828200 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:45.828206 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.836330 (+ 8124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.836334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:45.837367 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:45.837378 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.839010 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.839014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:45.839105 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:45.839128 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:03:45.839206 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 03:03:45.839279 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259}
I20260812 03:03:45.838541 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.838351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.5.96.130:60943)
0812 03:03:45.838440 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.838486 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 03:03:45.840263 17269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.819803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42226 (local address 127.5.96.129:44821)
0812 03:03:45.827845 (+ 8042us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:45.827849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:45.827864 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:03:45.827917 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:45.827920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:45.827969 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:45.828068 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:45.835466 (+ 7398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.836186 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.838325 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.838896 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.838999 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:45.839066 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:45.840080 (+ 1014us) server_negotiation.cc:299] Negotiation successful
0812 03:03:45.840163 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7955,"thread_start_us":79,"threads_started":1}
I20260812 03:03:45.840744 16562 tablet_service.cc:1467] Processing CreateTablet for tablet a7fe3e184a50446d9f08adec090d5628 (DEFAULT_TABLE table=test-table [id=ecc47971ba744b80b5fa9d193d8f55dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:45.840979 16562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7fe3e184a50446d9f08adec090d5628. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:45.843122 17289 tablet_bootstrap.cc:492] T a7fe3e184a50446d9f08adec090d5628 P ab340b69c0a246b4bfc2a120a824d8cd: Bootstrap starting.
I20260812 03:03:45.844095 17289 tablet_bootstrap.cc:654] T a7fe3e184a50446d9f08adec090d5628 P ab340b69c0a246b4bfc2a120a824d8cd: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:45.845319 17289 tablet_bootstrap.cc:492] T a7fe3e184a50446d9f08adec090d5628 P ab340b69c0a246b4bfc2a120a824d8cd: No bootstrap required, opened a new log
I20260812 03:03:45.845404 17289 ts_tablet_manager.cc:1397] T a7fe3e184a50446d9f08adec090d5628 P ab340b69c0a246b4bfc2a120a824d8cd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:45.845796 17289 raft_consensus.cc:348] T a7fe3e184a50446d9f08adec090d5628 P ab340b69c0a246b4bfc2a120a824d8cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19a80e5310d4f2396df01f7d289dc05" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 43647 } } peers { permanent_uuid: "ab340b69c0a246b4bfc2a120a824d8cd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 44821 } } peers { permanent_uuid: "fbc247aa6b9d4c308db729ea9b3c80a0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33861 } }
I20260812 03:03:45.845907 17289 raft_consensus.cc:374] T a7fe3e184a50446d9f08adec090d5628 P ab340b69c0a246b4bfc2a120a824d8cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:45.845944 17289 raft_consensus.cc:729] T a7fe3e184a50446d9f08adec090d5628 P ab340b69c0a246b4bfc2a120a824d8cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab340b69c0a246b4bfc2a120a824d8cd, State: Initialized, Role: FOLLOWER
I20260812 03:03:45.846064 17289 consensus_queue.cc:260] T a7fe3e184a50446d9f08adec090d5628 P ab340b69c0a246b4bfc2a120a824d8cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19a80e5310d4f2396df01f7d289dc05" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 43647 } } peers { permanent_uuid: "ab340b69c0a246b4bfc2a120a824d8cd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 44821 } } peers { permanent_uuid: "fbc247aa6b9d4c308db729ea9b3c80a0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33861 } }
I20260812 03:03:45.846377 17289 ts_tablet_manager.cc:1428] T a7fe3e184a50446d9f08adec090d5628 P ab340b69c0a246b4bfc2a120a824d8cd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:03:45.847769 16666 heartbeater.cc:503] Master 127.5.96.188:38451 was elected leader, sending a full tablet report...
I20260812 03:03:45.852006 17301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.846596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.5.96.129:56515)
0812 03:03:45.851848 (+ 5252us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.851911 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5176,"thread_start_us":103,"threads_started":1}
I20260812 03:03:45.852828 17301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.852598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.5.96.129:55743)
0812 03:03:45.852732 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.852779 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260812 03:03:45.876987 16892 tablet.cc:2369] T a7fe3e184a50446d9f08adec090d5628 P fbc247aa6b9d4c308db729ea9b3c80a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:03:45.947516 17279 raft_consensus.cc:482] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:03:45.947649 17279 raft_consensus.cc:504] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19a80e5310d4f2396df01f7d289dc05" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 43647 } } peers { permanent_uuid: "ab340b69c0a246b4bfc2a120a824d8cd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 44821 } } peers { permanent_uuid: "fbc247aa6b9d4c308db729ea9b3c80a0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33861 } }
I20260812 03:03:45.948730 17279 leader_election.cc:288] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab340b69c0a246b4bfc2a120a824d8cd (127.5.96.129:44821), fbc247aa6b9d4c308db729ea9b3c80a0 (127.5.96.130:33861)
I20260812 03:03:45.952217 17272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.949038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:38685 (local address 127.5.96.130:33861)
0812 03:03:45.949150 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:45.949153 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:45.949167 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:03:45.949334 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:45.949336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:45.949381 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:45.949441 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:45.949736 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.950229 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.951304 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.951708 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.951812 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:45.951866 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:45.952044 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 03:03:45.952116 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:03:45.952216 17280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.948930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:33861 (local address 127.5.96.131:38685)
0812 03:03:45.949099 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.949114 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:45.949217 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:45.949486 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:45.949489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:45.949508 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:45.949622 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:45.949627 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.950348 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.950352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:45.951160 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:45.951170 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.951826 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.951829 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:45.951929 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:45.951948 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 03:03:45.952015 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 03:03:45.952082 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260812 03:03:45.952320 17269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.949028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:53801 (local address 127.5.96.129:44821)
0812 03:03:45.949147 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:45.949152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:45.949463 (+ 311us) server_negotiation.cc:407] Connection header received
0812 03:03:45.949578 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:45.949581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:45.949627 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:45.949688 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:45.949942 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.950431 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.951379 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.951811 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.951914 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:45.951964 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:45.952128 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 03:03:45.952195 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 03:03:45.952462 17304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.948930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:44821 (local address 127.5.96.131:53801)
0812 03:03:45.949374 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.949385 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:45.949483 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:45.949707 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:45.949710 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:45.949733 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:45.949841 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:45.949846 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.950538 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.950541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:45.951258 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:45.951266 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.951914 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.951917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:45.952019 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:45.952038 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 03:03:45.952101 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 03:03:45.952168 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":150,"thread_start_us":136,"threads_started":1}
I20260812 03:03:45.952600 16811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7fe3e184a50446d9f08adec090d5628" candidate_uuid: "e19a80e5310d4f2396df01f7d289dc05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbc247aa6b9d4c308db729ea9b3c80a0" is_pre_election: true
I20260812 03:03:45.952737 16811 raft_consensus.cc:2454] T a7fe3e184a50446d9f08adec090d5628 P fbc247aa6b9d4c308db729ea9b3c80a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e19a80e5310d4f2396df01f7d289dc05 in term 0.
I20260812 03:03:45.953007 16592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7fe3e184a50446d9f08adec090d5628" candidate_uuid: "e19a80e5310d4f2396df01f7d289dc05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab340b69c0a246b4bfc2a120a824d8cd" is_pre_election: true
I20260812 03:03:45.953080 16927 leader_election.cc:302] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e19a80e5310d4f2396df01f7d289dc05, fbc247aa6b9d4c308db729ea9b3c80a0; no voters:
I20260812 03:03:45.953205 16592 raft_consensus.cc:2454] T a7fe3e184a50446d9f08adec090d5628 P ab340b69c0a246b4bfc2a120a824d8cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e19a80e5310d4f2396df01f7d289dc05 in term 0.
I20260812 03:03:45.953346 17279 raft_consensus.cc:2781] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:03:45.953405 17279 raft_consensus.cc:482] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:03:45.953450 17279 raft_consensus.cc:3037] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:45.954375 17279 raft_consensus.cc:504] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19a80e5310d4f2396df01f7d289dc05" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 43647 } } peers { permanent_uuid: "ab340b69c0a246b4bfc2a120a824d8cd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 44821 } } peers { permanent_uuid: "fbc247aa6b9d4c308db729ea9b3c80a0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33861 } }
I20260812 03:03:45.954677 17279 leader_election.cc:288] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05 [CANDIDATE]: Term 1 election: Requested vote from peers ab340b69c0a246b4bfc2a120a824d8cd (127.5.96.129:44821), fbc247aa6b9d4c308db729ea9b3c80a0 (127.5.96.130:33861)
I20260812 03:03:45.954989 16811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7fe3e184a50446d9f08adec090d5628" candidate_uuid: "e19a80e5310d4f2396df01f7d289dc05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbc247aa6b9d4c308db729ea9b3c80a0"
I20260812 03:03:45.954988 16592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7fe3e184a50446d9f08adec090d5628" candidate_uuid: "e19a80e5310d4f2396df01f7d289dc05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab340b69c0a246b4bfc2a120a824d8cd"
I20260812 03:03:45.955103 16592 raft_consensus.cc:3037] T a7fe3e184a50446d9f08adec090d5628 P ab340b69c0a246b4bfc2a120a824d8cd [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:45.955103 16811 raft_consensus.cc:3037] T a7fe3e184a50446d9f08adec090d5628 P fbc247aa6b9d4c308db729ea9b3c80a0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:45.956233 16811 raft_consensus.cc:2454] T a7fe3e184a50446d9f08adec090d5628 P fbc247aa6b9d4c308db729ea9b3c80a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e19a80e5310d4f2396df01f7d289dc05 in term 1.
I20260812 03:03:45.956233 16592 raft_consensus.cc:2454] T a7fe3e184a50446d9f08adec090d5628 P ab340b69c0a246b4bfc2a120a824d8cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e19a80e5310d4f2396df01f7d289dc05 in term 1.
I20260812 03:03:45.956490 16927 leader_election.cc:302] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e19a80e5310d4f2396df01f7d289dc05, fbc247aa6b9d4c308db729ea9b3c80a0; no voters:
I20260812 03:03:45.956648 17279 raft_consensus.cc:2781] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:45.956844 17279 raft_consensus.cc:686] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05 [term 1 LEADER]: Becoming Leader. State: Replica: e19a80e5310d4f2396df01f7d289dc05, State: Running, Role: LEADER
I20260812 03:03:45.957209 17279 consensus_queue.cc:237] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19a80e5310d4f2396df01f7d289dc05" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 43647 } } peers { permanent_uuid: "ab340b69c0a246b4bfc2a120a824d8cd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 44821 } } peers { permanent_uuid: "fbc247aa6b9d4c308db729ea9b3c80a0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33861 } }
I20260812 03:03:45.957471 17304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.957239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.5.96.131:57033)
0812 03:03:45.957335 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.957388 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 03:03:45.959446 16377 catalog_manager.cc:5495] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05 reported cstate change: term changed from 0 to 1, leader changed from <none> to e19a80e5310d4f2396df01f7d289dc05 (127.5.96.131). New cstate: current_term: 1 leader_uuid: "e19a80e5310d4f2396df01f7d289dc05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19a80e5310d4f2396df01f7d289dc05" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 43647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ab340b69c0a246b4bfc2a120a824d8cd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 44821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbc247aa6b9d4c308db729ea9b3c80a0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33861 } health_report { overall_health: UNKNOWN } } }
I20260812 03:03:45.977626 17270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.974650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45464 (local address 127.5.96.131:43647)
0812 03:03:45.974758 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:45.974762 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:45.974777 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:03:45.974942 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:45.974945 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:45.974993 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:45.975074 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:03:45.975289 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.975795 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.976527 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:45.976721 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:45.976787 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:45.976952 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:03:45.977361 (+ 409us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:03:45.977439 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:45.977472 (+ 33us) server_negotiation.cc:299] Negotiation successful
0812 03:03:45.977512 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:03:45.980028 16811 raft_consensus.cc:1261] T a7fe3e184a50446d9f08adec090d5628 P fbc247aa6b9d4c308db729ea9b3c80a0 [term 1 FOLLOWER]: Refusing update from remote peer e19a80e5310d4f2396df01f7d289dc05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:45.980350 16592 raft_consensus.cc:1261] T a7fe3e184a50446d9f08adec090d5628 P ab340b69c0a246b4bfc2a120a824d8cd [term 1 FOLLOWER]: Refusing update from remote peer e19a80e5310d4f2396df01f7d289dc05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:45.980554 17279 consensus_queue.cc:1035] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbc247aa6b9d4c308db729ea9b3c80a0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:45.980885 17305 consensus_queue.cc:1035] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab340b69c0a246b4bfc2a120a824d8cd" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:45.982028 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.981816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.5.96.130:36837)
0812 03:03:45.981913 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.981968 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 03:03:45.984179 17301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.983914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.5.96.129:51229)
0812 03:03:45.984058 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.984110 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260812 03:03:45.985209 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.984890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.5.96.130:41225)
0812 03:03:45.985108 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.985158 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260812 03:03:45.988081 17304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.987853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.5.96.131:48331)
0812 03:03:45.987966 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.988019 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 03:03:45.989478 17304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.988809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.5.96.131:46363)
0812 03:03:45.989367 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.989419 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":500}
I20260812 03:03:45.990482 17301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:45.990250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39179 (local address 127.5.96.129:59407)
0812 03:03:45.990365 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:45.990420 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:39179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 03:03:45.992380 16377 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:45.992501 16377 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:45.995929 16377 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5137614d58be42dbadab04e8e76d1a54: Sending DeleteTablet for 3 replicas of tablet a7fe3e184a50446d9f08adec090d5628
I20260812 03:03:45.996466 16994 tablet_service.cc:1514] Processing DeleteTablet for tablet a7fe3e184a50446d9f08adec090d5628 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:45 UTC) from {username='oryx', principal='oryx/127.5.96.188@KRBTEST.COM'} at 127.0.0.1:45452
I20260812 03:03:45.996749 16562 tablet_service.cc:1514] Processing DeleteTablet for tablet a7fe3e184a50446d9f08adec090d5628 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:45 UTC) from {username='oryx', principal='oryx/127.5.96.188@KRBTEST.COM'} at 127.0.0.1:42226
I20260812 03:03:45.997020 16789 tablet_service.cc:1514] Processing DeleteTablet for tablet a7fe3e184a50446d9f08adec090d5628 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:45 UTC) from {username='oryx', principal='oryx/127.5.96.188@KRBTEST.COM'} at 127.0.0.1:46166
I20260812 03:03:45.997044 17333 tablet_replica.cc:330] T a7fe3e184a50446d9f08adec090d5628 P ab340b69c0a246b4bfc2a120a824d8cd: stopping tablet replica
I20260812 03:03:45.997169 17333 raft_consensus.cc:2229] T a7fe3e184a50446d9f08adec090d5628 P ab340b69c0a246b4bfc2a120a824d8cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:45.997283 17333 raft_consensus.cc:2258] T a7fe3e184a50446d9f08adec090d5628 P ab340b69c0a246b4bfc2a120a824d8cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:45.997303 17334 tablet_replica.cc:330] T a7fe3e184a50446d9f08adec090d5628 P fbc247aa6b9d4c308db729ea9b3c80a0: stopping tablet replica
I20260812 03:03:45.997498 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 16316
I20260812 03:03:45.997411 17334 raft_consensus.cc:2229] T a7fe3e184a50446d9f08adec090d5628 P fbc247aa6b9d4c308db729ea9b3c80a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:45.998076 17334 raft_consensus.cc:2258] T a7fe3e184a50446d9f08adec090d5628 P fbc247aa6b9d4c308db729ea9b3c80a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:45.998816 17334 ts_tablet_manager.cc:1905] T a7fe3e184a50446d9f08adec090d5628 P fbc247aa6b9d4c308db729ea9b3c80a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:46.002619 17334 ts_tablet_manager.cc:1918] T a7fe3e184a50446d9f08adec090d5628 P fbc247aa6b9d4c308db729ea9b3c80a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:03:46.002733 17334 log.cc:1196] T a7fe3e184a50446d9f08adec090d5628 P fbc247aa6b9d4c308db729ea9b3c80a0: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-1/wal/wals/a7fe3e184a50446d9f08adec090d5628
I20260812 03:03:46.003052 17334 ts_tablet_manager.cc:1939] T a7fe3e184a50446d9f08adec090d5628 P fbc247aa6b9d4c308db729ea9b3c80a0: Deleting consensus metadata
I20260812 03:03:46.003844 16353 catalog_manager.cc:4841] TS fbc247aa6b9d4c308db729ea9b3c80a0 (127.5.96.130:33861): tablet a7fe3e184a50446d9f08adec090d5628 (table test-table [id=ecc47971ba744b80b5fa9d193d8f55dd]) successfully deleted
I20260812 03:03:46.015911 17332 tablet_replica.cc:330] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05: stopping tablet replica
I20260812 03:03:46.016055 17332 raft_consensus.cc:2229] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:03:46.016381 17332 raft_consensus.cc:2258] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:46.017047 17332 ts_tablet_manager.cc:1905] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:46.019580 17332 ts_tablet_manager.cc:1918] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:03:46.019682 17332 log.cc:1196] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786503790637001-5506-0/minicluster-data/ts-2/wal/wals/a7fe3e184a50446d9f08adec090d5628
I20260812 03:03:46.019999 17332 ts_tablet_manager.cc:1939] T a7fe3e184a50446d9f08adec090d5628 P e19a80e5310d4f2396df01f7d289dc05: Deleting consensus metadata
I20260812 03:03:46.020959 16349 catalog_manager.cc:4841] TS e19a80e5310d4f2396df01f7d289dc05 (127.5.96.131:43647): tablet a7fe3e184a50446d9f08adec090d5628 (table test-table [id=ecc47971ba744b80b5fa9d193d8f55dd]) successfully deleted
I20260812 03:03:46.021217 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 16679
I20260812 03:03:46.021534 17235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:46.021261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:44821 (local address 127.0.0.1:42242)
0812 03:03:46.021415 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:46.021462 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.129:44821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260812 03:03:46.030115 16347 catalog_manager.cc:4584] TS ab340b69c0a246b4bfc2a120a824d8cd (127.5.96.129:44821): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a7fe3e184a50446d9f08adec090d5628: Network error: Client connection negotiation failed: client connection to 127.5.96.129:44821: connect: Connection refused (error 111)
I20260812 03:03:46.036691 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 16896
I20260812 03:03:46.049062 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 15846
I20260812 03:03:46.064944 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 16011
2026-08-12T03:03:46Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5375 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:03:46 dist-test-slave-6bbx krb5kdc[17338](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:03:46 dist-test-slave-6bbx krb5kdc[17338](info): set up 2 sockets
Aug 12 03:03:46 dist-test-slave-6bbx krb5kdc[17338](info): commencing operation
krb5kdc: starting...
W20260812 03:03:47.233026 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:03:47 dist-test-slave-6bbx krb5kdc[17338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:03:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:03:47Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.190.keytab.
Entry for principal oryx/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.190.keytab.
I20260812 03:03:47.344103 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:35965
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.190.keytab
--principal=oryx/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:41897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:35965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:47.455638 17365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:47.455910 17365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:47.455966 17365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:47.459898 17365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:03:47.459944 17365 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 03:03:47.459966 17365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:47.459990 17365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:47.460021 17365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:03:47.460049 17365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:03:47.465122 17365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:41897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:35965
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.190.keytab
--principal=oryx/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:35965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:47.466039 17365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:47.466989 17365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:47.475049 17373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:47.475063 17372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:47.475061 17375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:47.475109 17365 server_base.cc:1034] running on GCE node
I20260812 03:03:47.475845 17365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:47.476809 17365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:47.477958 17365 hybrid_clock.cc:648] HybridClock initialized: now 1786503827477946 us; error 24 us; skew 500 ppm
Aug 12 03:03:47 dist-test-slave-6bbx krb5kdc[17338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503827, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:47.480589 17365 init.cc:377] Logged in from keytab as oryx/127.5.96.190@KRBTEST.COM (short username oryx)
I20260812 03:03:47.481376 17365 webserver.cc:460] Webserver started at http://127.5.96.190:41079/ using document root <none> and password file <none>
I20260812 03:03:47.481854 17365 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:47.481909 17365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:47.482066 17365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:47.483978 17365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "e5401c1b56aa4fe08ea1f21b4f3bfe88"
format_stamp: "Formatted at 2026-08-12 03:03:47 on dist-test-slave-6bbx"
server_key: "4310462aa31312a6d6e36a7e561ddd8f"
server_key_iv: "055edaf906a64baba7bf17ea622e544a"
server_key_version: "encryptionkey@0"
I20260812 03:03:47.484463 17365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "e5401c1b56aa4fe08ea1f21b4f3bfe88"
format_stamp: "Formatted at 2026-08-12 03:03:47 on dist-test-slave-6bbx"
server_key: "4310462aa31312a6d6e36a7e561ddd8f"
server_key_iv: "055edaf906a64baba7bf17ea622e544a"
server_key_version: "encryptionkey@0"
I20260812 03:03:47.487795 17365 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:03:47.489914 17387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:47.491011 17365 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:47.491120 17365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "e5401c1b56aa4fe08ea1f21b4f3bfe88"
format_stamp: "Formatted at 2026-08-12 03:03:47 on dist-test-slave-6bbx"
server_key: "4310462aa31312a6d6e36a7e561ddd8f"
server_key_iv: "055edaf906a64baba7bf17ea622e544a"
server_key_version: "encryptionkey@0"
I20260812 03:03:47.491204 17365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:47.506628 17365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:47.509420 17365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:47.509580 17365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:47.516934 17365 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:35965
I20260812 03:03:47.516973 17491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:35965 every 8 connection(s)
I20260812 03:03:47.517765 17365 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:03:47.519888 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 17365
I20260812 03:03:47.519986 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:03:47.520253 5506 external_mini_cluster.cc:1428] Setting key 693a6c008939388cfcc940547c37f7a5
I20260812 03:03:47.520393 17492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:47.527247 17492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5401c1b56aa4fe08ea1f21b4f3bfe88: Bootstrap starting.
Aug 12 03:03:47 dist-test-slave-6bbx krb5kdc[17338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM
I20260812 03:03:47.529424 17492 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5401c1b56aa4fe08ea1f21b4f3bfe88: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:47.530109 17492 log.cc:824] T 00000000000000000000000000000000 P e5401c1b56aa4fe08ea1f21b4f3bfe88: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:47.531805 17492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5401c1b56aa4fe08ea1f21b4f3bfe88: No bootstrap required, opened a new log
I20260812 03:03:47.534267 17499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:47.521519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51202 (local address 127.5.96.190:35965)
0812 03:03:47.521928 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:47.521946 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:47.521986 (+ 40us) server_negotiation.cc:407] Connection header received
0812 03:03:47.522940 (+ 954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:47.522964 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:47.523376 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:47.523746 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:47.524374 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:47.525820 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:47.526582 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:47.527245 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:47.527405 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:47.529592 (+ 2187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:47.529597 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:47.529608 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:47.529634 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:47.531793 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:47.532193 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:47.532197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:47.532202 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:47.532325 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:47.532577 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:47.532580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:47.532581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:47.532930 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:47.533109 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:47.533374 (+ 265us) server_negotiation.cc:299] Negotiation successful
0812 03:03:47.533620 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":112,"threads_started":1}
I20260812 03:03:47.534663 17492 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5401c1b56aa4fe08ea1f21b4f3bfe88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5401c1b56aa4fe08ea1f21b4f3bfe88" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35965 } }
I20260812 03:03:47.534873 17492 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5401c1b56aa4fe08ea1f21b4f3bfe88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:47.534951 17492 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5401c1b56aa4fe08ea1f21b4f3bfe88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5401c1b56aa4fe08ea1f21b4f3bfe88, State: Initialized, Role: FOLLOWER
I20260812 03:03:47.535408 17492 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5401c1b56aa4fe08ea1f21b4f3bfe88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5401c1b56aa4fe08ea1f21b4f3bfe88" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35965 } }
I20260812 03:03:47.535564 17492 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5401c1b56aa4fe08ea1f21b4f3bfe88 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:03:47.535632 17492 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5401c1b56aa4fe08ea1f21b4f3bfe88 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:03:47.535754 17492 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5401c1b56aa4fe08ea1f21b4f3bfe88 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:47.536664 17492 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5401c1b56aa4fe08ea1f21b4f3bfe88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5401c1b56aa4fe08ea1f21b4f3bfe88" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35965 } }
I20260812 03:03:47.536947 17492 leader_election.cc:302] T 00000000000000000000000000000000 P e5401c1b56aa4fe08ea1f21b4f3bfe88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5401c1b56aa4fe08ea1f21b4f3bfe88; no voters:
I20260812 03:03:47.537168 17492 leader_election.cc:288] T 00000000000000000000000000000000 P e5401c1b56aa4fe08ea1f21b4f3bfe88 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:03:47.537276 17501 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5401c1b56aa4fe08ea1f21b4f3bfe88 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:47.537462 17501 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5401c1b56aa4fe08ea1f21b4f3bfe88 [term 1 LEADER]: Becoming Leader. State: Replica: e5401c1b56aa4fe08ea1f21b4f3bfe88, State: Running, Role: LEADER
I20260812 03:03:47.537777 17501 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5401c1b56aa4fe08ea1f21b4f3bfe88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5401c1b56aa4fe08ea1f21b4f3bfe88" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35965 } }
I20260812 03:03:47.538206 17492 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5401c1b56aa4fe08ea1f21b4f3bfe88 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:03:47.539412 17503 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5401c1b56aa4fe08ea1f21b4f3bfe88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5401c1b56aa4fe08ea1f21b4f3bfe88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5401c1b56aa4fe08ea1f21b4f3bfe88" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35965 } } }
I20260812 03:03:47.539448 17504 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5401c1b56aa4fe08ea1f21b4f3bfe88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5401c1b56aa4fe08ea1f21b4f3bfe88. Latest consensus state: current_term: 1 leader_uuid: "e5401c1b56aa4fe08ea1f21b4f3bfe88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5401c1b56aa4fe08ea1f21b4f3bfe88" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35965 } } }
I20260812 03:03:47.539629 17504 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5401c1b56aa4fe08ea1f21b4f3bfe88 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:47.539975 17503 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5401c1b56aa4fe08ea1f21b4f3bfe88 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:47.540439 17520 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:03:47.543476 17520 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:03:47.548640 17520 catalog_manager.cc:1347] Generated new cluster ID: 99bd46e930c9497a83fb0e996b63e5ed
I20260812 03:03:47.548697 17520 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:03:47.557840 17520 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:03:47.558929 17520 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:03:47.568717 17520 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5401c1b56aa4fe08ea1f21b4f3bfe88: Generated new TSK 0
I20260812 03:03:47.569387 17520 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.96.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.96.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.96.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.129.keytab.
Entry for principal oryx/127.5.96.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.129.keytab.
I20260812 03:03:47.598464 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:0
--local_ip_for_outbound_sockets=127.5.96.129
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:35965
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.129.keytab
--principal=oryx/127.5.96.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:41897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:47.708016 17536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:47.708238 17536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:47.708277 17536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:47.712108 17536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:47.712153 17536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:47.712188 17536 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:03:47.712224 17536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:03:47.717387 17536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:41897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.129.keytab
--principal=oryx/127.5.96.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.96.190:35965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:47.718233 17536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:47.719099 17536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:47.727994 17549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:47.728003 17550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:47.727980 17552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:47.728309 17536 server_base.cc:1034] running on GCE node
I20260812 03:03:47.728672 17536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:47.729230 17536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:47.730387 17536 hybrid_clock.cc:648] HybridClock initialized: now 1786503827730373 us; error 31 us; skew 500 ppm
Aug 12 03:03:47 dist-test-slave-6bbx krb5kdc[17338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503827, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:47.732998 17536 init.cc:377] Logged in from keytab as oryx/127.5.96.129@KRBTEST.COM (short username oryx)
I20260812 03:03:47.733770 17536 webserver.cc:460] Webserver started at http://127.5.96.129:46117/ using document root <none> and password file <none>
I20260812 03:03:47.734266 17536 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:47.734325 17536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:47.734514 17536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:47.736428 17536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-0/data/instance:
uuid: "e907e57623154d839f025bac02fcc26e"
format_stamp: "Formatted at 2026-08-12 03:03:47 on dist-test-slave-6bbx"
server_key: "7366c07737ff68aab18fd69b7bc69ce5"
server_key_iv: "4d66aa1ac8c0a70e00c9bc16072f3310"
server_key_version: "encryptionkey@0"
I20260812 03:03:47.736919 17536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance:
uuid: "e907e57623154d839f025bac02fcc26e"
format_stamp: "Formatted at 2026-08-12 03:03:47 on dist-test-slave-6bbx"
server_key: "7366c07737ff68aab18fd69b7bc69ce5"
server_key_iv: "4d66aa1ac8c0a70e00c9bc16072f3310"
server_key_version: "encryptionkey@0"
I20260812 03:03:47.740412 17536 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:03:47.742626 17560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:47.743716 17536 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:03:47.743817 17536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "e907e57623154d839f025bac02fcc26e"
format_stamp: "Formatted at 2026-08-12 03:03:47 on dist-test-slave-6bbx"
server_key: "7366c07737ff68aab18fd69b7bc69ce5"
server_key_iv: "4d66aa1ac8c0a70e00c9bc16072f3310"
server_key_version: "encryptionkey@0"
I20260812 03:03:47.743906 17536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:47.756872 17536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:47.759573 17536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:47.759771 17536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:47.761094 17536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:47.761139 17536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:47.761174 17536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:47.761188 17536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:03:47 dist-test-slave-6bbx krb5kdc[17338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503827, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM
I20260812 03:03:47.770953 17536 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:39935
I20260812 03:03:47.771059 17753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:39935 every 8 connection(s)
I20260812 03:03:47.771966 17536 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:03:47.774021 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 17536
I20260812 03:03:47.774151 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance
I20260812 03:03:47.774299 17499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:47.761888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:57417 (local address 127.5.96.190:35965)
0812 03:03:47.762038 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:47.762043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:47.762814 (+ 771us) server_negotiation.cc:407] Connection header received
0812 03:03:47.763895 (+ 1081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:47.763898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:47.763959 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:47.764013 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:47.765290 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:47.765894 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:47.767521 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:47.767903 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:47.768009 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:47.770645 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:47.770649 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:47.770652 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:47.770679 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:47.772237 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:47.772896 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:47.772898 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:47.772900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:47.772978 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:47.773271 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:47.773273 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:47.773275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:47.773471 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:47.773563 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:47.774020 (+ 457us) server_negotiation.cc:299] Negotiation successful
0812 03:03:47.774131 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:03:47.774461 5506 external_mini_cluster.cc:1428] Setting key 594cea5d1dd542809ba5fcb151ecb6cf
I20260812 03:03:47.775173 17569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:47.762200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:35965 (local address 127.5.96.129:57417)
0812 03:03:47.762650 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:47.762697 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:47.763607 (+ 910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:47.764331 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:47.764337 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:47.764623 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:47.765137 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:47.765157 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:47.766011 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:47.766016 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:47.767394 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:47.767408 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:47.768077 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:47.768083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:47.768284 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:47.768777 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:47.768797 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:47.770490 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:47.772351 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:47.772357 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:47.772370 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:47.772674 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:47.773052 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:47.773055 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:47.773056 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:47.773184 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:47.773554 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:47.773558 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:47.773780 (+ 222us) client_negotiation.cc:769] Sending connection context
0812 03:03:47.773980 (+ 200us) client_negotiation.cc:240] Negotiation successful
0812 03:03:47.774300 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":101,"threads_started":1}
I20260812 03:03:47.778250 17754 heartbeater.cc:347] Connected to a master server at 127.5.96.190:35965
I20260812 03:03:47.778425 17754 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:47.778839 17754 heartbeater.cc:511] Master 127.5.96.190:35965 requested a full tablet report, sending...
I20260812 03:03:47.780329 17408 ts_manager.cc:194] Registered new tserver with Master: e907e57623154d839f025bac02fcc26e (127.5.96.129:39935)
I20260812 03:03:47.781747 17408 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.96.129@KRBTEST.COM'} at 127.5.96.129:57417
I20260812 03:03:47.785776 17764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:47.778605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:35965 (local address 127.5.96.129:52127)
0812 03:03:47.779354 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:47.779367 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:47.779463 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:47.779705 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:47.779708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:47.779906 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:47.780016 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:47.780023 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:47.780831 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:47.780834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:47.781367 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:47.781374 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:47.781788 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:47.781792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:47.781923 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:47.782345 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:47.782359 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:47.782702 (+ 343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:47.784544 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:47.784546 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:47.784547 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:47.784717 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:47.784975 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:47.784978 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:47.784979 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:47.785040 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:47.785403 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:47.785405 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:47.785471 (+ 66us) client_negotiation.cc:769] Sending connection context
0812 03:03:47.785540 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 03:03:47.785630 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"thread_start_us":82,"threads_started":1}
I20260812 03:03:47.785871 17499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:47.778656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:52127 (local address 127.5.96.190:35965)
0812 03:03:47.778771 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:47.778775 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:47.779429 (+ 654us) server_negotiation.cc:407] Connection header received
0812 03:03:47.779536 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:47.779539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:47.779583 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:47.779655 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:47.780129 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:47.780707 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:47.781469 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:47.781688 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:47.781769 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:47.782851 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:47.782855 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:47.782857 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:47.782884 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:47.784427 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:47.784810 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:47.784813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:47.784815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:47.784902 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:47.785112 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:47.785115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:47.785116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:47.785325 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:47.785425 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:47.785612 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 03:03:47.785724 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
WARNING: no policy specified for oryx/127.5.96.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.96.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.96.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.130.keytab.
Entry for principal oryx/127.5.96.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.130.keytab.
I20260812 03:03:47.801481 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:0
--local_ip_for_outbound_sockets=127.5.96.130
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:35965
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.130.keytab
--principal=oryx/127.5.96.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:41897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:47.909680 17768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:47.909907 17768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:47.909966 17768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:47.913808 17768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:47.913858 17768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:47.913910 17768 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:03:47.913964 17768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:03:47.918882 17768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:41897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.130.keytab
--principal=oryx/127.5.96.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.96.190:35965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:47.919797 17768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:47.920729 17768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:47.929598 17773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:47.929603 17777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:47.929625 17781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:47.929791 17768 server_base.cc:1034] running on GCE node
I20260812 03:03:47.930217 17768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:47.930804 17768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:47.931932 17768 hybrid_clock.cc:648] HybridClock initialized: now 1786503827931905 us; error 32 us; skew 500 ppm
Aug 12 03:03:47 dist-test-slave-6bbx krb5kdc[17338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503827, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:47.934552 17768 init.cc:377] Logged in from keytab as oryx/127.5.96.130@KRBTEST.COM (short username oryx)
I20260812 03:03:47.935294 17768 webserver.cc:460] Webserver started at http://127.5.96.130:39843/ using document root <none> and password file <none>
I20260812 03:03:47.935813 17768 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:47.935878 17768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:47.936082 17768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:47.937968 17768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-1/data/instance:
uuid: "744ee7761ce7492fb7667b74afe25d96"
format_stamp: "Formatted at 2026-08-12 03:03:47 on dist-test-slave-6bbx"
server_key: "42cbc3c0d86c3318e6217f8b6288511b"
server_key_iv: "f80ec3411385caa7fe30b75886cb2bfe"
server_key_version: "encryptionkey@0"
I20260812 03:03:47.938520 17768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance:
uuid: "744ee7761ce7492fb7667b74afe25d96"
format_stamp: "Formatted at 2026-08-12 03:03:47 on dist-test-slave-6bbx"
server_key: "42cbc3c0d86c3318e6217f8b6288511b"
server_key_iv: "f80ec3411385caa7fe30b75886cb2bfe"
server_key_version: "encryptionkey@0"
I20260812 03:03:47.942055 17768 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260812 03:03:47.944316 17789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:47.945379 17768 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260812 03:03:47.945473 17768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "744ee7761ce7492fb7667b74afe25d96"
format_stamp: "Formatted at 2026-08-12 03:03:47 on dist-test-slave-6bbx"
server_key: "42cbc3c0d86c3318e6217f8b6288511b"
server_key_iv: "f80ec3411385caa7fe30b75886cb2bfe"
server_key_version: "encryptionkey@0"
I20260812 03:03:47.945564 17768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:47.966331 17768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:47.968784 17768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:47.968935 17768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:47.970196 17768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:47.970244 17768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:47.970305 17768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:47.970333 17768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:03:47 dist-test-slave-6bbx krb5kdc[17338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503827, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM
I20260812 03:03:47.980495 17768 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:33917
I20260812 03:03:47.980767 17975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:33917 every 8 connection(s)
I20260812 03:03:47.981472 17768 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:03:47.983809 17499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:47.970948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:53853 (local address 127.5.96.190:35965)
0812 03:03:47.971063 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:47.971066 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:47.972026 (+ 960us) server_negotiation.cc:407] Connection header received
0812 03:03:47.972926 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:47.972929 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:47.972988 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:47.973070 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:47.974638 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:47.975447 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:47.977154 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:47.977479 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:47.977611 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:47.980226 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:47.980230 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:47.980233 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:47.980260 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:47.981802 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:47.982372 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:47.982376 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:47.982378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:47.982458 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:47.982775 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:47.982778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:47.982780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:47.982951 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:47.983036 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:47.983497 (+ 461us) server_negotiation.cc:299] Negotiation successful
0812 03:03:47.983616 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:03:47.984455 17804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:47.971432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:35965 (local address 127.5.96.130:53853)
0812 03:03:47.971889 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:47.971926 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:47.972714 (+ 788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:47.973363 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:47.973373 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:47.973840 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:47.974469 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:47.974490 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:47.975599 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:47.975603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:47.977028 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:47.977041 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:47.977640 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:47.977645 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:47.977841 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:47.978336 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:47.978357 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:47.980071 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:47.981911 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:47.981917 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:47.981930 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:47.982268 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:47.982540 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:47.982543 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:47.982545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:47.982672 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:47.983049 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:47.983055 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:47.983270 (+ 215us) client_negotiation.cc:769] Sending connection context
0812 03:03:47.983415 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 03:03:47.983727 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":141,"threads_started":1}
I20260812 03:03:47.985471 17977 heartbeater.cc:347] Connected to a master server at 127.5.96.190:35965
I20260812 03:03:47.985595 17977 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:47.986025 17977 heartbeater.cc:511] Master 127.5.96.190:35965 requested a full tablet report, sending...
I20260812 03:03:47.986704 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 17768
I20260812 03:03:47.986840 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance
I20260812 03:03:47.986941 17409 ts_manager.cc:194] Registered new tserver with Master: 744ee7761ce7492fb7667b74afe25d96 (127.5.96.130:33917)
I20260812 03:03:47.987146 5506 external_mini_cluster.cc:1428] Setting key 68e1e9eaf2461932cc0b55a148a27b31
I20260812 03:03:47.987553 17409 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.96.130@KRBTEST.COM'} at 127.5.96.130:53853
I20260812 03:03:47.994657 17499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:47.987557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:38697 (local address 127.5.96.190:35965)
0812 03:03:47.987656 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:47.987660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:47.988178 (+ 518us) server_negotiation.cc:407] Connection header received
0812 03:03:47.988271 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:47.988274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:47.988313 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:47.988374 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:47.988929 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:47.989655 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:47.990603 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:47.990851 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:47.990944 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:47.991860 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:47.991864 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:47.991867 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:47.991892 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:47.993159 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:47.993539 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:47.993542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:47.993544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:47.993617 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:47.993903 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:47.993907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:47.993908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:47.994097 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:47.994182 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:47.994434 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 03:03:47.994532 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 03:03:47.994673 17986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:47.987482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:35965 (local address 127.5.96.130:38697)
0812 03:03:47.988097 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:47.988110 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:47.988195 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:47.988398 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:47.988401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:47.988593 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:47.988698 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:47.988705 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:47.989782 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:47.989786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:47.990505 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:47.990516 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:47.990956 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:47.990959 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:47.991025 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:47.991405 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:47.991415 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:47.991705 (+ 290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:47.993268 (+ 1563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:47.993271 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:47.993273 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:47.993453 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:47.993699 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:47.993702 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:47.993704 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:47.993784 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:47.994190 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:47.994193 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:47.994284 (+ 91us) client_negotiation.cc:769] Sending connection context
0812 03:03:47.994368 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 03:03:47.994496 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":90,"threads_started":1}
WARNING: no policy specified for oryx/127.5.96.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.96.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.96.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.131.keytab.
Entry for principal oryx/127.5.96.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.131.keytab.
I20260812 03:03:48.013604 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:0
--local_ip_for_outbound_sockets=127.5.96.131
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:35965
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.131.keytab
--principal=oryx/127.5.96.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:41897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:48.121691 17990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:48.121932 17990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:48.121973 17990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:48.125870 17990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:48.125934 17990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:48.125985 17990 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:03:48.126040 17990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:03:48.130997 17990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:41897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.131.keytab
--principal=oryx/127.5.96.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.96.190:35965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:48.131959 17990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:48.132902 17990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:48.142491 18002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:48.142500 18001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:48.142532 18004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:48.142786 17990 server_base.cc:1034] running on GCE node
I20260812 03:03:48.143177 17990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:48.143778 17990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:48.144934 17990 hybrid_clock.cc:648] HybridClock initialized: now 1786503828144914 us; error 35 us; skew 500 ppm
Aug 12 03:03:48 dist-test-slave-6bbx krb5kdc[17338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503828, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:48.147475 17990 init.cc:377] Logged in from keytab as oryx/127.5.96.131@KRBTEST.COM (short username oryx)
I20260812 03:03:48.148329 17990 webserver.cc:460] Webserver started at http://127.5.96.131:34725/ using document root <none> and password file <none>
I20260812 03:03:48.148825 17990 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:48.148881 17990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:48.149087 17990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:48.150970 17990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-2/data/instance:
uuid: "b4002aff86fb431d968dc658fe8d8aab"
format_stamp: "Formatted at 2026-08-12 03:03:48 on dist-test-slave-6bbx"
server_key: "52fe23902c393866f95abd3ff833eba9"
server_key_iv: "63bfdf5bd67b1c8d7e3f756affab2759"
server_key_version: "encryptionkey@0"
I20260812 03:03:48.151482 17990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance:
uuid: "b4002aff86fb431d968dc658fe8d8aab"
format_stamp: "Formatted at 2026-08-12 03:03:48 on dist-test-slave-6bbx"
server_key: "52fe23902c393866f95abd3ff833eba9"
server_key_iv: "63bfdf5bd67b1c8d7e3f756affab2759"
server_key_version: "encryptionkey@0"
I20260812 03:03:48.154974 17990 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:03:48.157258 18015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:48.158368 17990 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:03:48.158469 17990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "b4002aff86fb431d968dc658fe8d8aab"
format_stamp: "Formatted at 2026-08-12 03:03:48 on dist-test-slave-6bbx"
server_key: "52fe23902c393866f95abd3ff833eba9"
server_key_iv: "63bfdf5bd67b1c8d7e3f756affab2759"
server_key_version: "encryptionkey@0"
I20260812 03:03:48.158569 17990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:48.170548 17990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:48.172977 17990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:48.173108 17990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:48.174396 17990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:48.174444 17990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:48.174481 17990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:48.174508 17990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:03:48 dist-test-slave-6bbx krb5kdc[17338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503828, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM
I20260812 03:03:48.184865 17990 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:40781
I20260812 03:03:48.184983 18205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:40781 every 8 connection(s)
I20260812 03:03:48.185825 17990 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:03:48.187256 17499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.175052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:48849 (local address 127.5.96.190:35965)
0812 03:03:48.175206 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:48.175212 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:48.176011 (+ 799us) server_negotiation.cc:407] Connection header received
0812 03:03:48.177136 (+ 1125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:48.177140 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:48.177195 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:48.177293 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:48.178464 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.179159 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.180380 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.180607 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.180696 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.183607 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:48.183611 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:48.183614 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:48.183641 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:48.185122 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:48.185648 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:48.185651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:48.185652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:48.185755 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:48.186097 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:48.186100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:48.186101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:48.186318 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:48.186419 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:48.187029 (+ 610us) server_negotiation.cc:299] Negotiation successful
0812 03:03:48.187134 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:03:48.188007 18030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.175373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:35965 (local address 127.5.96.131:48849)
0812 03:03:48.175859 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:48.175910 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:48.176917 (+ 1007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:48.177492 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:48.177501 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:48.177891 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:48.178323 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.178335 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.179279 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.179283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.180263 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.180273 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.180750 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.180755 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.180908 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.181441 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:48.181461 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:48.183449 (+ 1988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:48.185225 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:48.185231 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:48.185243 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:48.185556 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:48.185844 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:48.185847 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:48.185849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:48.186010 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:48.186474 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:48.186481 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:48.186788 (+ 307us) client_negotiation.cc:769] Sending connection context
0812 03:03:48.186958 (+ 170us) client_negotiation.cc:240] Negotiation successful
0812 03:03:48.187184 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":100,"threads_started":1}
I20260812 03:03:48.189015 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 17990
I20260812 03:03:48.189117 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance
I20260812 03:03:48.189101 18206 heartbeater.cc:347] Connected to a master server at 127.5.96.190:35965
I20260812 03:03:48.189213 18206 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:48.189406 5506 external_mini_cluster.cc:1428] Setting key 78d409ba0613124cd3709715d219c183
I20260812 03:03:48.189738 18206 heartbeater.cc:511] Master 127.5.96.190:35965 requested a full tablet report, sending...
I20260812 03:03:48.190757 17409 ts_manager.cc:194] Registered new tserver with Master: b4002aff86fb431d968dc658fe8d8aab (127.5.96.131:40781)
I20260812 03:03:48.191430 17409 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.96.131@KRBTEST.COM'} at 127.5.96.131:48849
I20260812 03:03:48.197157 18216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.191164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:35965 (local address 127.5.96.131:51233)
0812 03:03:48.191449 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:48.191463 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:48.191556 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:48.191863 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:48.191866 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:48.192065 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:48.192175 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.192182 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.192895 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.192898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.193572 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.193581 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.194022 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.194025 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.194096 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.194494 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:48.194508 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:48.194788 (+ 280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:48.196109 (+ 1321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:48.196111 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:48.196113 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:48.196292 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:48.196482 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:48.196484 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:48.196485 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:48.196548 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:48.196812 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:48.196813 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:48.196879 (+ 66us) client_negotiation.cc:769] Sending connection context
0812 03:03:48.196948 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 03:03:48.197039 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":130,"threads_started":1}
I20260812 03:03:48.197173 17499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.191269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:51233 (local address 127.5.96.190:35965)
0812 03:03:48.191378 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:48.191382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:48.191531 (+ 149us) server_negotiation.cc:407] Connection header received
0812 03:03:48.191622 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:48.191626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:48.191687 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:48.191751 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:48.192287 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.192769 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.193667 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.193934 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.194003 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.194884 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:48.194887 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:48.194889 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:48.194906 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:48.196013 (+ 1107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:48.196358 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:48.196360 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:48.196361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:48.196426 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:48.196603 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:48.196605 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:48.196606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:48.196750 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:48.196818 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:48.196990 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 03:03:48.197071 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:03:48.203330 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 03:03:48 dist-test-slave-6bbx krb5kdc[17338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503828, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:03:48 dist-test-slave-6bbx krb5kdc[17338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:48 dist-test-slave-6bbx krb5kdc[17338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 03:03:48.226387 17499 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:48.221249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47782 (local address 127.5.96.190:35965)
0812 03:03:48.221350 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:48.221354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:48.221366 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:03:48.221395 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:48.221397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:48.221430 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:48.221489 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:48.222083 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.222576 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.223321 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.223517 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.223581 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.226265 (+ 2684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47782: BlockingRecv error: recv got EOF from 127.0.0.1:47782 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 12 03:03:48 dist-test-slave-6bbx krb5kdc[17338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503828, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM
I20260812 03:03:48.234503 17499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.227560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47790 (local address 127.5.96.190:35965)
0812 03:03:48.227698 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:48.227701 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:48.227804 (+ 103us) server_negotiation.cc:407] Connection header received
0812 03:03:48.227906 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:48.227909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:48.227946 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:48.228009 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:48.228556 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.229026 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.229761 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.229969 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.230050 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.231975 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:48.231979 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:48.231982 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:48.232005 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:48.233099 (+ 1094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:48.233466 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:48.233470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:48.233471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:48.233544 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:48.233785 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:48.233788 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:48.233790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:48.233949 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:48.234060 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:48.234271 (+ 211us) server_negotiation.cc:299] Negotiation successful
0812 03:03:48.234370 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:03:48.237028 17409 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47790:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:03:48.240072 17409 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:03:48.253194 18245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.248144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:33917 (local address 127.0.0.1:41028)
0812 03:03:48.248597 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:48.248610 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:48.248695 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:48.249450 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:48.249453 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:48.249471 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:48.249557 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.249563 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.250898 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.250902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.251816 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.251825 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.252700 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.252704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.252810 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.252847 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 03:03:48.252949 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 03:03:48.253063 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":35,"thread_start_us":92,"threads_started":1}
I20260812 03:03:48.253213 18247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.248666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41028 (local address 127.5.96.130:33917)
0812 03:03:48.248993 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:48.249006 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:48.249030 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:03:48.249096 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:48.249101 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:48.249298 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:48.249700 (+ 402us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:48.249834 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.250798 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.251943 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.252586 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.252684 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.252857 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:48.252999 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 03:03:48.253089 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":110,"threads_started":1}
I20260812 03:03:48.253500 18241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.247763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:40781 (local address 127.0.0.1:33962)
0812 03:03:48.248130 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:48.248176 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:48.248302 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:48.249048 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:48.249054 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:48.249076 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:48.249186 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.249192 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.250623 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.250626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.251911 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.251922 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.253129 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.253132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.253216 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.253240 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:03:48.253308 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 03:03:48.253376 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":77,"threads_started":1}
I20260812 03:03:48.253993 18242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.247915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33962 (local address 127.5.96.131:40781)
0812 03:03:48.248367 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:48.248379 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:48.248403 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:03:48.248474 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:48.248480 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:48.248743 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:48.248889 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:48.249385 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.250514 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.252042 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.252838 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.252931 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.253069 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:48.253822 (+ 753us) server_negotiation.cc:299] Negotiation successful
0812 03:03:48.253892 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":92,"threads_started":1}
I20260812 03:03:48.254983 18246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.248644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:39935 (local address 127.0.0.1:36112)
0812 03:03:48.249499 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:48.249513 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:48.249600 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:48.250100 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:48.250103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:48.250123 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:48.250202 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.250207 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.251374 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.251378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.252574 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.252584 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.254623 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.254626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.254692 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.254711 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 03:03:48.254792 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 03:03:48.254858 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":750,"mutex_wait_us":28,"thread_start_us":97,"threads_started":1}
I20260812 03:03:48.255424 18248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.249127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36112 (local address 127.5.96.129:39935)
0812 03:03:48.249646 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:48.249661 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:48.249682 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:03:48.249744 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:48.249749 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:48.249905 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:48.250117 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:48.250288 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.251239 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.252708 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.253436 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.253535 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.253700 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:48.255196 (+ 1496us) server_negotiation.cc:299] Negotiation successful
0812 03:03:48.255288 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":80,"threads_started":1}
I20260812 03:03:48.256176 17874 tablet_service.cc:1467] Processing CreateTablet for tablet 8013b66e6e3943b4a51cf4809a30f992 (DEFAULT_TABLE table=test-table [id=3109c198f7cd4e3a966a4fe10ce8271b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:48.256527 18090 tablet_service.cc:1467] Processing CreateTablet for tablet 8013b66e6e3943b4a51cf4809a30f992 (DEFAULT_TABLE table=test-table [id=3109c198f7cd4e3a966a4fe10ce8271b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:48.257314 17874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8013b66e6e3943b4a51cf4809a30f992. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:48.257452 18090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8013b66e6e3943b4a51cf4809a30f992. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:48.261665 17637 tablet_service.cc:1467] Processing CreateTablet for tablet 8013b66e6e3943b4a51cf4809a30f992 (DEFAULT_TABLE table=test-table [id=3109c198f7cd4e3a966a4fe10ce8271b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:48.262655 17637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8013b66e6e3943b4a51cf4809a30f992. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:48.265121 18250 tablet_bootstrap.cc:492] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96: Bootstrap starting.
I20260812 03:03:48.265544 18252 tablet_bootstrap.cc:492] T 8013b66e6e3943b4a51cf4809a30f992 P b4002aff86fb431d968dc658fe8d8aab: Bootstrap starting.
I20260812 03:03:48.267902 18252 tablet_bootstrap.cc:654] T 8013b66e6e3943b4a51cf4809a30f992 P b4002aff86fb431d968dc658fe8d8aab: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:48.268812 18252 log.cc:824] T 8013b66e6e3943b4a51cf4809a30f992 P b4002aff86fb431d968dc658fe8d8aab: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:48.269465 18250 tablet_bootstrap.cc:654] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:48.269778 18256 tablet_bootstrap.cc:492] T 8013b66e6e3943b4a51cf4809a30f992 P e907e57623154d839f025bac02fcc26e: Bootstrap starting.
I20260812 03:03:48.270179 18250 log.cc:824] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:48.271585 18256 tablet_bootstrap.cc:654] T 8013b66e6e3943b4a51cf4809a30f992 P e907e57623154d839f025bac02fcc26e: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:48.271687 18252 tablet_bootstrap.cc:492] T 8013b66e6e3943b4a51cf4809a30f992 P b4002aff86fb431d968dc658fe8d8aab: No bootstrap required, opened a new log
I20260812 03:03:48.271880 18252 ts_tablet_manager.cc:1397] T 8013b66e6e3943b4a51cf4809a30f992 P b4002aff86fb431d968dc658fe8d8aab: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 03:03:48.272176 18250 tablet_bootstrap.cc:492] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96: No bootstrap required, opened a new log
I20260812 03:03:48.272296 18256 log.cc:824] T 8013b66e6e3943b4a51cf4809a30f992 P e907e57623154d839f025bac02fcc26e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:48.272357 18250 ts_tablet_manager.cc:1397] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 03:03:48.273761 18256 tablet_bootstrap.cc:492] T 8013b66e6e3943b4a51cf4809a30f992 P e907e57623154d839f025bac02fcc26e: No bootstrap required, opened a new log
I20260812 03:03:48.273922 18256 ts_tablet_manager.cc:1397] T 8013b66e6e3943b4a51cf4809a30f992 P e907e57623154d839f025bac02fcc26e: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260812 03:03:48.276072 18250 raft_consensus.cc:348] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } } peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } }
I20260812 03:03:48.276103 18252 raft_consensus.cc:348] T 8013b66e6e3943b4a51cf4809a30f992 P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } } peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } }
I20260812 03:03:48.276331 18252 raft_consensus.cc:374] T 8013b66e6e3943b4a51cf4809a30f992 P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:48.276365 18250 raft_consensus.cc:374] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:48.276407 18252 raft_consensus.cc:729] T 8013b66e6e3943b4a51cf4809a30f992 P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4002aff86fb431d968dc658fe8d8aab, State: Initialized, Role: FOLLOWER
I20260812 03:03:48.276441 18250 raft_consensus.cc:729] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 744ee7761ce7492fb7667b74afe25d96, State: Initialized, Role: FOLLOWER
I20260812 03:03:48.276885 18250 consensus_queue.cc:260] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } } peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } }
I20260812 03:03:48.276885 18252 consensus_queue.cc:260] T 8013b66e6e3943b4a51cf4809a30f992 P b4002aff86fb431d968dc658fe8d8aab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } } peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } }
I20260812 03:03:48.277135 18256 raft_consensus.cc:348] T 8013b66e6e3943b4a51cf4809a30f992 P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } } peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } }
I20260812 03:03:48.277482 18256 raft_consensus.cc:374] T 8013b66e6e3943b4a51cf4809a30f992 P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:48.277581 18256 raft_consensus.cc:729] T 8013b66e6e3943b4a51cf4809a30f992 P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e907e57623154d839f025bac02fcc26e, State: Initialized, Role: FOLLOWER
I20260812 03:03:48.277813 18206 heartbeater.cc:503] Master 127.5.96.190:35965 was elected leader, sending a full tablet report...
I20260812 03:03:48.277873 18252 ts_tablet_manager.cc:1428] T 8013b66e6e3943b4a51cf4809a30f992 P b4002aff86fb431d968dc658fe8d8aab: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260812 03:03:48.277873 18250 ts_tablet_manager.cc:1428] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 03:03:48.278120 17977 heartbeater.cc:503] Master 127.5.96.190:35965 was elected leader, sending a full tablet report...
I20260812 03:03:48.278129 18256 consensus_queue.cc:260] T 8013b66e6e3943b4a51cf4809a30f992 P e907e57623154d839f025bac02fcc26e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } } peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } }
I20260812 03:03:48.278705 17754 heartbeater.cc:503] Master 127.5.96.190:35965 was elected leader, sending a full tablet report...
I20260812 03:03:48.279050 18256 ts_tablet_manager.cc:1428] T 8013b66e6e3943b4a51cf4809a30f992 P e907e57623154d839f025bac02fcc26e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 03:03:48.401681 18267 raft_consensus.cc:482] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:03:48.401894 18267 raft_consensus.cc:504] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } } peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } }
I20260812 03:03:48.403046 18267 leader_election.cc:288] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4002aff86fb431d968dc658fe8d8aab (127.5.96.131:40781), e907e57623154d839f025bac02fcc26e (127.5.96.129:39935)
I20260812 03:03:48.406739 17804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.403269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:40781 (local address 127.5.96.130:58491)
0812 03:03:48.403436 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:48.403452 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:48.403547 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:48.403802 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:48.403805 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:48.403819 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:48.403920 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.403925 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.404802 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.404806 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.405705 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.405715 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.406347 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.406351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.406449 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.406476 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 03:03:48.406556 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 03:03:48.406628 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260812 03:03:48.406764 18242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.403434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:58491 (local address 127.5.96.131:40781)
0812 03:03:48.403573 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:48.403578 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:48.403592 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:03:48.403638 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:48.403640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:48.403716 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:48.403788 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:48.404041 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.404688 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.405839 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.406229 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.406323 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.406367 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:48.406569 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 03:03:48.406636 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:03:48.407486 18122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8013b66e6e3943b4a51cf4809a30f992" candidate_uuid: "744ee7761ce7492fb7667b74afe25d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4002aff86fb431d968dc658fe8d8aab" is_pre_election: true
I20260812 03:03:48.407632 18248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.403420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:45033 (local address 127.5.96.129:39935)
0812 03:03:48.403543 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:48.403547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:48.404268 (+ 721us) server_negotiation.cc:407] Connection header received
0812 03:03:48.404345 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:48.404347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:48.404397 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:48.404468 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:48.405088 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.405551 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.406611 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.407144 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.407255 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.407292 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:48.407487 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 03:03:48.407540 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:03:48.407709 18269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.403269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:39935 (local address 127.5.96.130:45033)
0812 03:03:48.404173 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:48.404184 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:48.404271 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:48.404476 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:48.404478 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:48.404488 (+ 10us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:48.404968 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.404976 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.405659 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.405662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.406502 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.406510 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.407282 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.407286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.407379 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.407402 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:03:48.407478 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 03:03:48.407545 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":845,"mutex_wait_us":581,"thread_start_us":99,"threads_started":1}
I20260812 03:03:48.407855 18122 raft_consensus.cc:2454] T 8013b66e6e3943b4a51cf4809a30f992 P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 744ee7761ce7492fb7667b74afe25d96 in term 0.
I20260812 03:03:48.408280 17673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8013b66e6e3943b4a51cf4809a30f992" candidate_uuid: "744ee7761ce7492fb7667b74afe25d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e907e57623154d839f025bac02fcc26e" is_pre_election: true
I20260812 03:03:48.408316 17792 leader_election.cc:302] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 744ee7761ce7492fb7667b74afe25d96, b4002aff86fb431d968dc658fe8d8aab; no voters:
I20260812 03:03:48.408557 17673 raft_consensus.cc:2454] T 8013b66e6e3943b4a51cf4809a30f992 P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 744ee7761ce7492fb7667b74afe25d96 in term 0.
I20260812 03:03:48.408557 18267 raft_consensus.cc:2781] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:03:48.408644 18267 raft_consensus.cc:482] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:03:48.408696 18267 raft_consensus.cc:3037] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:48.409701 18267 raft_consensus.cc:504] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } } peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } }
I20260812 03:03:48.409994 18267 leader_election.cc:288] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96 [CANDIDATE]: Term 1 election: Requested vote from peers b4002aff86fb431d968dc658fe8d8aab (127.5.96.131:40781), e907e57623154d839f025bac02fcc26e (127.5.96.129:39935)
I20260812 03:03:48.410352 18122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8013b66e6e3943b4a51cf4809a30f992" candidate_uuid: "744ee7761ce7492fb7667b74afe25d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4002aff86fb431d968dc658fe8d8aab"
I20260812 03:03:48.410375 17673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8013b66e6e3943b4a51cf4809a30f992" candidate_uuid: "744ee7761ce7492fb7667b74afe25d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e907e57623154d839f025bac02fcc26e"
I20260812 03:03:48.410468 18122 raft_consensus.cc:3037] T 8013b66e6e3943b4a51cf4809a30f992 P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:48.410478 17673 raft_consensus.cc:3037] T 8013b66e6e3943b4a51cf4809a30f992 P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:48.411453 17673 raft_consensus.cc:2454] T 8013b66e6e3943b4a51cf4809a30f992 P e907e57623154d839f025bac02fcc26e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 744ee7761ce7492fb7667b74afe25d96 in term 1.
I20260812 03:03:48.411778 17795 leader_election.cc:302] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 744ee7761ce7492fb7667b74afe25d96, e907e57623154d839f025bac02fcc26e; no voters:
I20260812 03:03:48.411823 18122 raft_consensus.cc:2454] T 8013b66e6e3943b4a51cf4809a30f992 P b4002aff86fb431d968dc658fe8d8aab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 744ee7761ce7492fb7667b74afe25d96 in term 1.
I20260812 03:03:48.412027 18267 raft_consensus.cc:2781] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:48.412241 18267 raft_consensus.cc:686] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96 [term 1 LEADER]: Becoming Leader. State: Replica: 744ee7761ce7492fb7667b74afe25d96, State: Running, Role: LEADER
I20260812 03:03:48.412567 18267 consensus_queue.cc:237] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } } peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } }
I20260812 03:03:48.415532 17409 catalog_manager.cc:5495] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 744ee7761ce7492fb7667b74afe25d96 (127.5.96.130). New cstate: current_term: 1 leader_uuid: "744ee7761ce7492fb7667b74afe25d96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } health_report { overall_health: HEALTHY } } }
W20260812 03:03:48.436911 18209 tablet.cc:2369] T 8013b66e6e3943b4a51cf4809a30f992 P b4002aff86fb431d968dc658fe8d8aab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:03:48.437788 18247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.434666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41030 (local address 127.5.96.130:33917)
0812 03:03:48.434785 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:48.434789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:48.434802 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:03:48.434922 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:48.434924 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:48.434972 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:48.435052 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:03:48.435229 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.435731 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.436488 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.436694 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.436762 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.437009 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:03:48.437431 (+ 422us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:03:48.437513 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:48.437600 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 03:03:48.437647 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:03:48.443241 18122 raft_consensus.cc:1261] T 8013b66e6e3943b4a51cf4809a30f992 P b4002aff86fb431d968dc658fe8d8aab [term 1 FOLLOWER]: Refusing update from remote peer 744ee7761ce7492fb7667b74afe25d96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:48.443241 17673 raft_consensus.cc:1261] T 8013b66e6e3943b4a51cf4809a30f992 P e907e57623154d839f025bac02fcc26e [term 1 FOLLOWER]: Refusing update from remote peer 744ee7761ce7492fb7667b74afe25d96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:48.443889 18267 consensus_queue.cc:1035] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:48.444099 18270 consensus_queue.cc:1035] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:48.457806 18283 mvcc.cc:205] Tried to move back new op lower bound from 7317519681297317888 to 7317519681180254208. Current Snapshot: MvccSnapshot[applied={T|T < 7317519681297317888}]
I20260812 03:03:48.458165 17409 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:48.458336 17409 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:48.460911 17409 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e5401c1b56aa4fe08ea1f21b4f3bfe88: Sending DeleteTablet for 3 replicas of tablet 8013b66e6e3943b4a51cf4809a30f992
I20260812 03:03:48.461609 17874 tablet_service.cc:1514] Processing DeleteTablet for tablet 8013b66e6e3943b4a51cf4809a30f992 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:48 UTC) from {username='oryx', principal='oryx/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:41028
I20260812 03:03:48.461609 18090 tablet_service.cc:1514] Processing DeleteTablet for tablet 8013b66e6e3943b4a51cf4809a30f992 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:48 UTC) from {username='oryx', principal='oryx/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:33962
I20260812 03:03:48.462061 17637 tablet_service.cc:1514] Processing DeleteTablet for tablet 8013b66e6e3943b4a51cf4809a30f992 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:48 UTC) from {username='oryx', principal='oryx/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:36112
I20260812 03:03:48.462507 18293 tablet_replica.cc:330] T 8013b66e6e3943b4a51cf4809a30f992 P e907e57623154d839f025bac02fcc26e: stopping tablet replica
I20260812 03:03:48.462677 18292 tablet_replica.cc:330] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96: stopping tablet replica
I20260812 03:03:48.462839 18293 raft_consensus.cc:2229] T 8013b66e6e3943b4a51cf4809a30f992 P e907e57623154d839f025bac02fcc26e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:48.463121 18293 raft_consensus.cc:2258] T 8013b66e6e3943b4a51cf4809a30f992 P e907e57623154d839f025bac02fcc26e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:48.463413 17409 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47790:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:03:48.463802 17409 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:03:48.463985 18291 tablet_replica.cc:330] T 8013b66e6e3943b4a51cf4809a30f992 P b4002aff86fb431d968dc658fe8d8aab: stopping tablet replica
I20260812 03:03:48.464290 18291 raft_consensus.cc:2229] T 8013b66e6e3943b4a51cf4809a30f992 P b4002aff86fb431d968dc658fe8d8aab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:48.464588 18291 raft_consensus.cc:2258] T 8013b66e6e3943b4a51cf4809a30f992 P b4002aff86fb431d968dc658fe8d8aab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:48.463265 18292 raft_consensus.cc:2229] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:03:48.465063 18293 ts_tablet_manager.cc:1905] T 8013b66e6e3943b4a51cf4809a30f992 P e907e57623154d839f025bac02fcc26e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:48.467437 18291 ts_tablet_manager.cc:1905] T 8013b66e6e3943b4a51cf4809a30f992 P b4002aff86fb431d968dc658fe8d8aab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:48.467774 18293 ts_tablet_manager.cc:1918] T 8013b66e6e3943b4a51cf4809a30f992 P e907e57623154d839f025bac02fcc26e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:03:48.467868 18293 log.cc:1196] T 8013b66e6e3943b4a51cf4809a30f992 P e907e57623154d839f025bac02fcc26e: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-0/wal/wals/8013b66e6e3943b4a51cf4809a30f992
I20260812 03:03:48.468137 18293 ts_tablet_manager.cc:1939] T 8013b66e6e3943b4a51cf4809a30f992 P e907e57623154d839f025bac02fcc26e: Deleting consensus metadata
I20260812 03:03:48.468583 18090 tablet_service.cc:1467] Processing CreateTablet for tablet 98db608a9e1d4569bbda5c963cdbffcd (DEFAULT_TABLE table=test-table [id=5322a9d315e54b4da6a8dd837c090b03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:48.468837 18090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98db608a9e1d4569bbda5c963cdbffcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:48.468861 17637 tablet_service.cc:1467] Processing CreateTablet for tablet 98db608a9e1d4569bbda5c963cdbffcd (DEFAULT_TABLE table=test-table [id=5322a9d315e54b4da6a8dd837c090b03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:48.468534 17874 tablet_service.cc:1467] Processing CreateTablet for tablet 98db608a9e1d4569bbda5c963cdbffcd (DEFAULT_TABLE table=test-table [id=5322a9d315e54b4da6a8dd837c090b03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:48.469470 17874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98db608a9e1d4569bbda5c963cdbffcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:48.471050 18252 tablet_bootstrap.cc:492] T 98db608a9e1d4569bbda5c963cdbffcd P b4002aff86fb431d968dc658fe8d8aab: Bootstrap starting.
I20260812 03:03:48.471776 18250 tablet_bootstrap.cc:492] T 98db608a9e1d4569bbda5c963cdbffcd P 744ee7761ce7492fb7667b74afe25d96: Bootstrap starting.
I20260812 03:03:48.472031 18252 tablet_bootstrap.cc:654] T 98db608a9e1d4569bbda5c963cdbffcd P b4002aff86fb431d968dc658fe8d8aab: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:48.472589 18250 tablet_bootstrap.cc:654] T 98db608a9e1d4569bbda5c963cdbffcd P 744ee7761ce7492fb7667b74afe25d96: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:48.472930 17394 catalog_manager.cc:4841] TS e907e57623154d839f025bac02fcc26e (127.5.96.129:39935): tablet 8013b66e6e3943b4a51cf4809a30f992 (table test-table [id=3109c198f7cd4e3a966a4fe10ce8271b]) successfully deleted
I20260812 03:03:48.473145 18252 tablet_bootstrap.cc:492] T 98db608a9e1d4569bbda5c963cdbffcd P b4002aff86fb431d968dc658fe8d8aab: No bootstrap required, opened a new log
I20260812 03:03:48.473246 18252 ts_tablet_manager.cc:1397] T 98db608a9e1d4569bbda5c963cdbffcd P b4002aff86fb431d968dc658fe8d8aab: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:48.473477 17637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98db608a9e1d4569bbda5c963cdbffcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:48.473685 18252 raft_consensus.cc:348] T 98db608a9e1d4569bbda5c963cdbffcd P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:48.473798 18252 raft_consensus.cc:374] T 98db608a9e1d4569bbda5c963cdbffcd P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:48.473841 18252 raft_consensus.cc:729] T 98db608a9e1d4569bbda5c963cdbffcd P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4002aff86fb431d968dc658fe8d8aab, State: Initialized, Role: FOLLOWER
I20260812 03:03:48.473963 18252 consensus_queue.cc:260] T 98db608a9e1d4569bbda5c963cdbffcd P b4002aff86fb431d968dc658fe8d8aab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:48.474196 18252 ts_tablet_manager.cc:1428] T 98db608a9e1d4569bbda5c963cdbffcd P b4002aff86fb431d968dc658fe8d8aab: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 03:03:48.475627 18256 tablet_bootstrap.cc:492] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e: Bootstrap starting.
I20260812 03:03:48.476372 18256 tablet_bootstrap.cc:654] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:48.466336 18292 raft_consensus.cc:2258] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:48.477538 18256 tablet_bootstrap.cc:492] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e: No bootstrap required, opened a new log
I20260812 03:03:48.477638 18256 ts_tablet_manager.cc:1397] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 03:03:48.478356 18256 raft_consensus.cc:348] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:48.478459 18256 raft_consensus.cc:374] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:48.478489 18256 raft_consensus.cc:729] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e907e57623154d839f025bac02fcc26e, State: Initialized, Role: FOLLOWER
I20260812 03:03:48.478595 18256 consensus_queue.cc:260] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:48.478819 18256 ts_tablet_manager.cc:1428] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 03:03:48.478845 18292 ts_tablet_manager.cc:1905] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:48.481272 18292 ts_tablet_manager.cc:1918] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:03:48.481354 18292 log.cc:1196] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-1/wal/wals/8013b66e6e3943b4a51cf4809a30f992
I20260812 03:03:48.481603 18292 ts_tablet_manager.cc:1939] T 8013b66e6e3943b4a51cf4809a30f992 P 744ee7761ce7492fb7667b74afe25d96: Deleting consensus metadata
I20260812 03:03:48.482524 17390 catalog_manager.cc:4841] TS 744ee7761ce7492fb7667b74afe25d96 (127.5.96.130:33917): tablet 8013b66e6e3943b4a51cf4809a30f992 (table test-table [id=3109c198f7cd4e3a966a4fe10ce8271b]) successfully deleted
I20260812 03:03:48.484282 18291 ts_tablet_manager.cc:1918] T 8013b66e6e3943b4a51cf4809a30f992 P b4002aff86fb431d968dc658fe8d8aab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:03:48.484362 18291 log.cc:1196] T 8013b66e6e3943b4a51cf4809a30f992 P b4002aff86fb431d968dc658fe8d8aab: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-2/wal/wals/8013b66e6e3943b4a51cf4809a30f992
I20260812 03:03:48.484589 18291 ts_tablet_manager.cc:1939] T 8013b66e6e3943b4a51cf4809a30f992 P b4002aff86fb431d968dc658fe8d8aab: Deleting consensus metadata
I20260812 03:03:48.485548 17392 catalog_manager.cc:4841] TS b4002aff86fb431d968dc658fe8d8aab (127.5.96.131:40781): tablet 8013b66e6e3943b4a51cf4809a30f992 (table test-table [id=3109c198f7cd4e3a966a4fe10ce8271b]) successfully deleted
I20260812 03:03:48.486151 18250 tablet_bootstrap.cc:492] T 98db608a9e1d4569bbda5c963cdbffcd P 744ee7761ce7492fb7667b74afe25d96: No bootstrap required, opened a new log
I20260812 03:03:48.486230 18250 ts_tablet_manager.cc:1397] T 98db608a9e1d4569bbda5c963cdbffcd P 744ee7761ce7492fb7667b74afe25d96: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.001s
I20260812 03:03:48.486696 18250 raft_consensus.cc:348] T 98db608a9e1d4569bbda5c963cdbffcd P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:48.486801 18250 raft_consensus.cc:374] T 98db608a9e1d4569bbda5c963cdbffcd P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:48.486831 18250 raft_consensus.cc:729] T 98db608a9e1d4569bbda5c963cdbffcd P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 744ee7761ce7492fb7667b74afe25d96, State: Initialized, Role: FOLLOWER
I20260812 03:03:48.486948 18250 consensus_queue.cc:260] T 98db608a9e1d4569bbda5c963cdbffcd P 744ee7761ce7492fb7667b74afe25d96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:48.487190 18250 ts_tablet_manager.cc:1428] T 98db608a9e1d4569bbda5c963cdbffcd P 744ee7761ce7492fb7667b74afe25d96: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
W20260812 03:03:48.523307 17755 tablet.cc:2369] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:03:48.614396 18268 raft_consensus.cc:482] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:03:48.614547 18268 raft_consensus.cc:504] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:48.615590 18268 leader_election.cc:288] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 744ee7761ce7492fb7667b74afe25d96 (127.5.96.130:33917), b4002aff86fb431d968dc658fe8d8aab (127.5.96.131:40781)
I20260812 03:03:48.619449 18242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.615969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:44917 (local address 127.5.96.131:40781)
0812 03:03:48.616166 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:48.616170 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:48.616331 (+ 161us) server_negotiation.cc:407] Connection header received
0812 03:03:48.616441 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:48.616444 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:48.616487 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:48.616559 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:48.616833 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.617308 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.618491 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.618874 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.618963 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.619010 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:48.619259 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 03:03:48.619332 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260812 03:03:48.619450 18247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.615933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:35815 (local address 127.5.96.130:33917)
0812 03:03:48.616054 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:48.616059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:48.616265 (+ 206us) server_negotiation.cc:407] Connection header received
0812 03:03:48.616373 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:48.616376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:48.616424 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:48.616484 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:48.616819 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.617318 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.618512 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.618930 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.619052 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.619119 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:48.619255 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 03:03:48.619326 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:03:48.619473 18300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.615888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:33917 (local address 127.5.96.129:35815)
0812 03:03:48.616151 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:48.616167 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:48.616284 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:48.616512 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:48.616515 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:48.616531 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:48.616704 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.616710 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.617426 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.617429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.618330 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.618339 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.619058 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.619062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.619136 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.619156 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 03:03:48.619218 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 03:03:48.619323 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":72,"thread_start_us":95,"threads_started":1}
I20260812 03:03:48.619482 18301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.615905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:40781 (local address 127.5.96.129:44917)
0812 03:03:48.616222 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:48.616236 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:48.616347 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:48.616586 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:48.616589 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:48.616605 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:48.616719 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.616724 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.617410 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.617413 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.618325 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.618335 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.619010 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.619014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.619123 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.619152 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 03:03:48.619225 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 03:03:48.619312 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":62,"thread_start_us":115,"threads_started":1}
I20260812 03:03:48.619993 17906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98db608a9e1d4569bbda5c963cdbffcd" candidate_uuid: "e907e57623154d839f025bac02fcc26e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "744ee7761ce7492fb7667b74afe25d96" is_pre_election: true
I20260812 03:03:48.620200 18122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98db608a9e1d4569bbda5c963cdbffcd" candidate_uuid: "e907e57623154d839f025bac02fcc26e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4002aff86fb431d968dc658fe8d8aab" is_pre_election: true
I20260812 03:03:48.620245 17906 raft_consensus.cc:2454] T 98db608a9e1d4569bbda5c963cdbffcd P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e907e57623154d839f025bac02fcc26e in term 0.
I20260812 03:03:48.620338 18122 raft_consensus.cc:2454] T 98db608a9e1d4569bbda5c963cdbffcd P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e907e57623154d839f025bac02fcc26e in term 0.
I20260812 03:03:48.620651 17562 leader_election.cc:302] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4002aff86fb431d968dc658fe8d8aab, e907e57623154d839f025bac02fcc26e; no voters:
I20260812 03:03:48.620919 18268 raft_consensus.cc:2781] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:03:48.620976 18268 raft_consensus.cc:482] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:03:48.621001 18268 raft_consensus.cc:3037] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:48.622200 18268 raft_consensus.cc:504] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:48.622536 18268 leader_election.cc:288] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e [CANDIDATE]: Term 1 election: Requested vote from peers 744ee7761ce7492fb7667b74afe25d96 (127.5.96.130:33917), b4002aff86fb431d968dc658fe8d8aab (127.5.96.131:40781)
I20260812 03:03:48.622869 17906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98db608a9e1d4569bbda5c963cdbffcd" candidate_uuid: "e907e57623154d839f025bac02fcc26e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "744ee7761ce7492fb7667b74afe25d96"
I20260812 03:03:48.622866 18122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98db608a9e1d4569bbda5c963cdbffcd" candidate_uuid: "e907e57623154d839f025bac02fcc26e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4002aff86fb431d968dc658fe8d8aab"
I20260812 03:03:48.622987 18122 raft_consensus.cc:3037] T 98db608a9e1d4569bbda5c963cdbffcd P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:48.622987 17906 raft_consensus.cc:3037] T 98db608a9e1d4569bbda5c963cdbffcd P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:48.624126 17906 raft_consensus.cc:2454] T 98db608a9e1d4569bbda5c963cdbffcd P 744ee7761ce7492fb7667b74afe25d96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e907e57623154d839f025bac02fcc26e in term 1.
I20260812 03:03:48.624126 18122 raft_consensus.cc:2454] T 98db608a9e1d4569bbda5c963cdbffcd P b4002aff86fb431d968dc658fe8d8aab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e907e57623154d839f025bac02fcc26e in term 1.
I20260812 03:03:48.624408 17562 leader_election.cc:302] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4002aff86fb431d968dc658fe8d8aab, e907e57623154d839f025bac02fcc26e; no voters:
I20260812 03:03:48.624619 18268 raft_consensus.cc:2781] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:48.624817 18268 raft_consensus.cc:686] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e [term 1 LEADER]: Becoming Leader. State: Replica: e907e57623154d839f025bac02fcc26e, State: Running, Role: LEADER
I20260812 03:03:48.625051 18268 consensus_queue.cc:237] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:48.627136 17409 catalog_manager.cc:5495] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e reported cstate change: term changed from 0 to 1, leader changed from <none> to e907e57623154d839f025bac02fcc26e (127.5.96.129). New cstate: current_term: 1 leader_uuid: "e907e57623154d839f025bac02fcc26e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } health_report { overall_health: UNKNOWN } } }
Aug 12 03:03:48 dist-test-slave-6bbx krb5kdc[17338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503827, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.190@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM
I20260812 03:03:48.700855 17499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.693394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47798 (local address 127.5.96.190:35965)
0812 03:03:48.693519 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:48.693524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:48.693677 (+ 153us) server_negotiation.cc:407] Connection header received
0812 03:03:48.693809 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:48.693812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:48.693850 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:48.693913 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:48.694619 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.695138 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.695952 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.696161 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.696230 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.698260 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:48.698263 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:48.698265 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:48.698284 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:48.699357 (+ 1073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:48.699833 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:48.699836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:48.699837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:48.699905 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:48.700098 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:48.700100 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:48.700101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:48.700247 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:48.700313 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:48.700577 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 03:03:48.700720 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":42}
I20260812 03:03:48.700856 18320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.693337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:35965 (local address 127.0.0.1:47798)
0812 03:03:48.693600 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:48.693611 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:48.693729 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:48.693941 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:48.693944 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:48.694393 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:48.694514 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.694521 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.695217 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.695220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.695852 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.695862 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.696238 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.696240 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.696311 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.696818 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:48.696830 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:48.698145 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:48.699452 (+ 1307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:48.699462 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:48.699468 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:48.699762 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:48.699968 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:48.699969 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:48.699971 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:48.700042 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:48.700315 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:48.700319 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:48.700416 (+ 97us) client_negotiation.cc:769] Sending connection context
0812 03:03:48.700508 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 03:03:48.700656 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":3,"thread_start_us":94,"threads_started":1}
I20260812 03:03:48.704480 17409 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:47798:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 03:03:48.705245 17409 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:03:48.709694 17637 tablet_service.cc:1467] Processing CreateTablet for tablet ff378d516f234c1f83789bd2db207607 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a85c329837d244d6a258cdb7a6d33a30]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:03:48.709806 17874 tablet_service.cc:1467] Processing CreateTablet for tablet ff378d516f234c1f83789bd2db207607 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a85c329837d244d6a258cdb7a6d33a30]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:03:48.709856 18090 tablet_service.cc:1467] Processing CreateTablet for tablet ff378d516f234c1f83789bd2db207607 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a85c329837d244d6a258cdb7a6d33a30]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:03:48.710098 17637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff378d516f234c1f83789bd2db207607. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:48.710178 17874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff378d516f234c1f83789bd2db207607. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:48.710178 18090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff378d516f234c1f83789bd2db207607. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:48.712495 18256 tablet_bootstrap.cc:492] T ff378d516f234c1f83789bd2db207607 P e907e57623154d839f025bac02fcc26e: Bootstrap starting.
I20260812 03:03:48.712905 18252 tablet_bootstrap.cc:492] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab: Bootstrap starting.
I20260812 03:03:48.713434 18256 tablet_bootstrap.cc:654] T ff378d516f234c1f83789bd2db207607 P e907e57623154d839f025bac02fcc26e: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:48.713759 18252 tablet_bootstrap.cc:654] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:48.714098 18250 tablet_bootstrap.cc:492] T ff378d516f234c1f83789bd2db207607 P 744ee7761ce7492fb7667b74afe25d96: Bootstrap starting.
I20260812 03:03:48.714598 18256 tablet_bootstrap.cc:492] T ff378d516f234c1f83789bd2db207607 P e907e57623154d839f025bac02fcc26e: No bootstrap required, opened a new log
I20260812 03:03:48.714658 18256 ts_tablet_manager.cc:1397] T ff378d516f234c1f83789bd2db207607 P e907e57623154d839f025bac02fcc26e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 03:03:48.714751 18250 tablet_bootstrap.cc:654] T ff378d516f234c1f83789bd2db207607 P 744ee7761ce7492fb7667b74afe25d96: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:48.714990 18256 raft_consensus.cc:348] T ff378d516f234c1f83789bd2db207607 P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:48.715068 18256 raft_consensus.cc:374] T ff378d516f234c1f83789bd2db207607 P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:48.715096 18256 raft_consensus.cc:729] T ff378d516f234c1f83789bd2db207607 P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e907e57623154d839f025bac02fcc26e, State: Initialized, Role: FOLLOWER
I20260812 03:03:48.715211 18256 consensus_queue.cc:260] T ff378d516f234c1f83789bd2db207607 P e907e57623154d839f025bac02fcc26e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:48.715493 18256 ts_tablet_manager.cc:1428] T ff378d516f234c1f83789bd2db207607 P e907e57623154d839f025bac02fcc26e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:03:48.715466 18268 tablet_replica.cc:439] T ff378d516f234c1f83789bd2db207607 P e907e57623154d839f025bac02fcc26e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } } }
I20260812 03:03:48.715595 18268 tablet_replica.cc:442] T ff378d516f234c1f83789bd2db207607 P e907e57623154d839f025bac02fcc26e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:03:48.715868 18250 tablet_bootstrap.cc:492] T ff378d516f234c1f83789bd2db207607 P 744ee7761ce7492fb7667b74afe25d96: No bootstrap required, opened a new log
I20260812 03:03:48.715957 18250 ts_tablet_manager.cc:1397] T ff378d516f234c1f83789bd2db207607 P 744ee7761ce7492fb7667b74afe25d96: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260812 03:03:48.716341 18252 tablet_bootstrap.cc:492] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab: No bootstrap required, opened a new log
I20260812 03:03:48.716415 18252 ts_tablet_manager.cc:1397] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 03:03:48.716355 18250 raft_consensus.cc:348] T ff378d516f234c1f83789bd2db207607 P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:48.716449 18250 raft_consensus.cc:374] T ff378d516f234c1f83789bd2db207607 P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:48.716480 18250 raft_consensus.cc:729] T ff378d516f234c1f83789bd2db207607 P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 744ee7761ce7492fb7667b74afe25d96, State: Initialized, Role: FOLLOWER
I20260812 03:03:48.716598 18250 consensus_queue.cc:260] T ff378d516f234c1f83789bd2db207607 P 744ee7761ce7492fb7667b74afe25d96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:48.716833 18250 ts_tablet_manager.cc:1428] T ff378d516f234c1f83789bd2db207607 P 744ee7761ce7492fb7667b74afe25d96: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:03:48.716827 18252 raft_consensus.cc:348] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:48.716923 18252 raft_consensus.cc:374] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:48.716955 18252 raft_consensus.cc:729] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4002aff86fb431d968dc658fe8d8aab, State: Initialized, Role: FOLLOWER
I20260812 03:03:48.717016 18287 tablet_replica.cc:439] T ff378d516f234c1f83789bd2db207607 P 744ee7761ce7492fb7667b74afe25d96: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } } }
I20260812 03:03:48.717067 18252 consensus_queue.cc:260] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:48.717128 18287 tablet_replica.cc:442] T ff378d516f234c1f83789bd2db207607 P 744ee7761ce7492fb7667b74afe25d96: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:03:48.717314 18252 ts_tablet_manager.cc:1428] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:03:48.717526 18266 tablet_replica.cc:439] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } } }
I20260812 03:03:48.717643 18266 tablet_replica.cc:442] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab: This TxnStatusTablet replica's current role is: FOLLOWER
W20260812 03:03:48.732776 17981 tablet.cc:2369] T 98db608a9e1d4569bbda5c963cdbffcd P 744ee7761ce7492fb7667b74afe25d96: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:03:48.823012 18266 raft_consensus.cc:482] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:03:48.823144 18266 raft_consensus.cc:504] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:48.824282 18266 leader_election.cc:288] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e907e57623154d839f025bac02fcc26e (127.5.96.129:39935), 744ee7761ce7492fb7667b74afe25d96 (127.5.96.130:33917)
I20260812 03:03:48.828096 18248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.824591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:51297 (local address 127.5.96.129:39935)
0812 03:03:48.824710 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:48.824715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:48.824904 (+ 189us) server_negotiation.cc:407] Connection header received
0812 03:03:48.825018 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:48.825021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:48.825067 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:48.825139 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:48.825429 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.826001 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.827178 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.827566 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.827691 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.827757 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:48.827909 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 03:03:48.827975 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:03:48.828135 18326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.824510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:39935 (local address 127.5.96.131:51297)
0812 03:03:48.824817 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:48.824831 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:48.824926 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:48.825154 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:48.825157 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:48.825189 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:48.825327 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.825335 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.826094 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.826098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.827052 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.827077 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.827695 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.827698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.827793 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.827817 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:03:48.827887 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 03:03:48.827959 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":65,"thread_start_us":88,"threads_started":1}
I20260812 03:03:48.828126 18247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.824651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:35149 (local address 127.5.96.130:33917)
0812 03:03:48.824785 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:48.824790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:48.825028 (+ 238us) server_negotiation.cc:407] Connection header received
0812 03:03:48.825120 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:48.825123 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:48.825173 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:48.825248 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:03:48.825494 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.825971 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.827167 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.827597 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.827720 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.827778 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:48.827936 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 03:03:48.828007 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:03:48.828145 18324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.824521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:33917 (local address 127.5.96.131:35149)
0812 03:03:48.824936 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:48.824949 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:48.825042 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:48.825277 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:48.825279 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:48.825293 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:03:48.825394 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.825399 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.826073 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.826076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.827066 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.827076 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.827721 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.827725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.827824 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.827846 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:03:48.827913 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 03:03:48.827984 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":70,"thread_start_us":71,"threads_started":1}
I20260812 03:03:48.828510 17673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff378d516f234c1f83789bd2db207607" candidate_uuid: "b4002aff86fb431d968dc658fe8d8aab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e907e57623154d839f025bac02fcc26e" is_pre_election: true
I20260812 03:03:48.828655 17673 raft_consensus.cc:2454] T ff378d516f234c1f83789bd2db207607 P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4002aff86fb431d968dc658fe8d8aab in term 0.
I20260812 03:03:48.828707 17906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff378d516f234c1f83789bd2db207607" candidate_uuid: "b4002aff86fb431d968dc658fe8d8aab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "744ee7761ce7492fb7667b74afe25d96" is_pre_election: true
I20260812 03:03:48.828819 17906 raft_consensus.cc:2454] T ff378d516f234c1f83789bd2db207607 P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4002aff86fb431d968dc658fe8d8aab in term 0.
I20260812 03:03:48.829015 18022 leader_election.cc:302] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4002aff86fb431d968dc658fe8d8aab, e907e57623154d839f025bac02fcc26e; no voters:
I20260812 03:03:48.829280 18266 raft_consensus.cc:2781] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:03:48.829337 18266 raft_consensus.cc:482] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:03:48.829362 18266 raft_consensus.cc:3037] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:48.830304 18266 raft_consensus.cc:504] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:48.830610 18266 leader_election.cc:288] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab [CANDIDATE]: Term 1 election: Requested vote from peers e907e57623154d839f025bac02fcc26e (127.5.96.129:39935), 744ee7761ce7492fb7667b74afe25d96 (127.5.96.130:33917)
I20260812 03:03:48.830933 17673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff378d516f234c1f83789bd2db207607" candidate_uuid: "b4002aff86fb431d968dc658fe8d8aab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e907e57623154d839f025bac02fcc26e"
I20260812 03:03:48.830942 17906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff378d516f234c1f83789bd2db207607" candidate_uuid: "b4002aff86fb431d968dc658fe8d8aab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "744ee7761ce7492fb7667b74afe25d96"
I20260812 03:03:48.831053 17673 raft_consensus.cc:3037] T ff378d516f234c1f83789bd2db207607 P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:48.831053 17906 raft_consensus.cc:3037] T ff378d516f234c1f83789bd2db207607 P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:48.832147 17906 raft_consensus.cc:2454] T ff378d516f234c1f83789bd2db207607 P 744ee7761ce7492fb7667b74afe25d96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4002aff86fb431d968dc658fe8d8aab in term 1.
I20260812 03:03:48.832147 17673 raft_consensus.cc:2454] T ff378d516f234c1f83789bd2db207607 P e907e57623154d839f025bac02fcc26e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4002aff86fb431d968dc658fe8d8aab in term 1.
I20260812 03:03:48.832408 18022 leader_election.cc:302] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4002aff86fb431d968dc658fe8d8aab, e907e57623154d839f025bac02fcc26e; no voters:
I20260812 03:03:48.832576 18266 raft_consensus.cc:2781] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:48.832767 18266 raft_consensus.cc:686] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab [term 1 LEADER]: Becoming Leader. State: Replica: b4002aff86fb431d968dc658fe8d8aab, State: Running, Role: LEADER
I20260812 03:03:48.832990 18266 consensus_queue.cc:237] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:48.834122 18327 tablet_replica.cc:439] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab: TxnStatusTablet state changed. Reason: New leader b4002aff86fb431d968dc658fe8d8aab. Latest consensus state: current_term: 1 leader_uuid: "b4002aff86fb431d968dc658fe8d8aab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } } }
I20260812 03:03:48.834271 18327 tablet_replica.cc:442] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:03:48.834697 18329 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 03:03:48.834852 18329 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 03:03:48.835181 17407 catalog_manager.cc:5495] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab reported cstate change: term changed from 0 to 1, leader changed from <none> to b4002aff86fb431d968dc658fe8d8aab (127.5.96.131). New cstate: current_term: 1 leader_uuid: "b4002aff86fb431d968dc658fe8d8aab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } health_report { overall_health: HEALTHY } } }
I20260812 03:03:48.873206 18242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.869384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33976 (local address 127.5.96.131:40781)
0812 03:03:48.869494 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:48.869497 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:48.869603 (+ 106us) server_negotiation.cc:407] Connection header received
0812 03:03:48.869794 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:48.869797 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:48.869845 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:48.869921 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:03:48.870160 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.870632 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.871952 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.872158 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.872231 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.872517 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:03:48.872889 (+ 372us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:03:48.872967 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:48.873062 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 03:03:48.873104 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:03:48.873278 18320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.869290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:40781 (local address 127.0.0.1:33976)
0812 03:03:48.869532 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:48.869547 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:48.869672 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:48.869951 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:48.869954 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:48.869963 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:03:48.870071 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.870078 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.870722 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.870725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.871838 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.871851 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.872254 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.872258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.872343 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.872397 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:03:48.872972 (+ 575us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:03:48.872984 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 03:03:48.873053 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 03:03:48.873139 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142}
I20260812 03:03:48.875922 17673 raft_consensus.cc:1261] T ff378d516f234c1f83789bd2db207607 P e907e57623154d839f025bac02fcc26e [term 1 FOLLOWER]: Refusing update from remote peer b4002aff86fb431d968dc658fe8d8aab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:48.876216 17906 raft_consensus.cc:1261] T ff378d516f234c1f83789bd2db207607 P 744ee7761ce7492fb7667b74afe25d96 [term 1 FOLLOWER]: Refusing update from remote peer b4002aff86fb431d968dc658fe8d8aab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:48.876363 18327 consensus_queue.cc:1035] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:48.876760 18266 consensus_queue.cc:1035] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab [LEADER]: Connected to new peer: Peer: permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:48.877892 18268 tablet_replica.cc:439] T ff378d516f234c1f83789bd2db207607 P e907e57623154d839f025bac02fcc26e: TxnStatusTablet state changed. Reason: New leader b4002aff86fb431d968dc658fe8d8aab. Latest consensus state: current_term: 1 leader_uuid: "b4002aff86fb431d968dc658fe8d8aab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } } }
I20260812 03:03:48.878005 18268 tablet_replica.cc:442] T ff378d516f234c1f83789bd2db207607 P e907e57623154d839f025bac02fcc26e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:03:48.878300 18287 tablet_replica.cc:439] T ff378d516f234c1f83789bd2db207607 P 744ee7761ce7492fb7667b74afe25d96: TxnStatusTablet state changed. Reason: New leader b4002aff86fb431d968dc658fe8d8aab. Latest consensus state: current_term: 1 leader_uuid: "b4002aff86fb431d968dc658fe8d8aab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } } }
I20260812 03:03:48.878404 18287 tablet_replica.cc:442] T ff378d516f234c1f83789bd2db207607 P 744ee7761ce7492fb7667b74afe25d96: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:03:48.879983 18278 mvcc.cc:205] Tried to move back new op lower bound from 7317519683070730240 to 7317519682902044672. Current Snapshot: MvccSnapshot[applied={T|T < 7317519683070730240}]
I20260812 03:03:48.880089 18266 tablet_replica.cc:439] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b4002aff86fb431d968dc658fe8d8aab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } } }
I20260812 03:03:48.880201 18266 tablet_replica.cc:442] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:03:48.881011 18266 tablet_replica.cc:439] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b4002aff86fb431d968dc658fe8d8aab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } } }
I20260812 03:03:48.881117 18266 tablet_replica.cc:442] T ff378d516f234c1f83789bd2db207607 P b4002aff86fb431d968dc658fe8d8aab: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:03:48.881217 18268 tablet_replica.cc:439] T ff378d516f234c1f83789bd2db207607 P e907e57623154d839f025bac02fcc26e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b4002aff86fb431d968dc658fe8d8aab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } } }
I20260812 03:03:48.881332 18268 tablet_replica.cc:442] T ff378d516f234c1f83789bd2db207607 P e907e57623154d839f025bac02fcc26e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:03:48.882704 18287 tablet_replica.cc:439] T ff378d516f234c1f83789bd2db207607 P 744ee7761ce7492fb7667b74afe25d96: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b4002aff86fb431d968dc658fe8d8aab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } } }
I20260812 03:03:48.882813 18287 tablet_replica.cc:442] T ff378d516f234c1f83789bd2db207607 P 744ee7761ce7492fb7667b74afe25d96: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:03:48.882892 18275 mvcc.cc:205] Tried to move back new op lower bound from 7317519683070730240 to 7317519682902044672. Current Snapshot: MvccSnapshot[applied={T|T < 7317519683070730240 or (T in {7317519683070730240})}]
I20260812 03:03:48.889562 18248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.886208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36126 (local address 127.5.96.129:39935)
0812 03:03:48.886321 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:48.886324 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:48.886387 (+ 63us) server_negotiation.cc:407] Connection header received
0812 03:03:48.886544 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:48.886547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:48.886601 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:48.886680 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:03:48.886911 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.887394 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.888236 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.888462 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.888547 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.888808 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:03:48.889220 (+ 412us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:03:48.889321 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:48.889394 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 03:03:48.889446 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 03:03:48.899364 18343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.896090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:40781 (local address 127.5.96.129:36439)
0812 03:03:48.896371 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:48.896382 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:48.896462 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:48.896669 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:48.896671 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:48.896675 (+ 4us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:03:48.896758 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.896762 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.897667 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.897669 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.898260 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.898267 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.898789 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.898791 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.898856 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.898893 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:03:48.899141 (+ 248us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:03:48.899144 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:03:48.899184 (+ 40us) client_negotiation.cc:240] Negotiation successful
0812 03:03:48.899239 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":90,"threads_started":1}
I20260812 03:03:48.899363 18242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.896185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:36439 (local address 127.5.96.131:40781)
0812 03:03:48.896295 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:48.896299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:48.896447 (+ 148us) server_negotiation.cc:407] Connection header received
0812 03:03:48.896538 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:48.896541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:48.896585 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:48.896664 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:03:48.896843 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.897581 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.898364 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.898696 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.898794 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.899001 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:03:48.899072 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:03:48.899139 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:48.899200 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 03:03:48.899246 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:03:48.905346 18122 raft_consensus.cc:1261] T 98db608a9e1d4569bbda5c963cdbffcd P b4002aff86fb431d968dc658fe8d8aab [term 1 FOLLOWER]: Refusing update from remote peer e907e57623154d839f025bac02fcc26e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:48.905364 17906 raft_consensus.cc:1261] T 98db608a9e1d4569bbda5c963cdbffcd P 744ee7761ce7492fb7667b74afe25d96 [term 1 FOLLOWER]: Refusing update from remote peer e907e57623154d839f025bac02fcc26e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:48.905732 18303 consensus_queue.cc:1035] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e [LEADER]: Connected to new peer: Peer: permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:48.906085 18268 consensus_queue.cc:1035] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:48.926720 18248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.923035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:50645 (local address 127.5.96.129:39935)
0812 03:03:48.923133 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:48.923137 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:48.923330 (+ 193us) server_negotiation.cc:407] Connection header received
0812 03:03:48.923427 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:48.923429 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:48.923465 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:48.923520 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:03:48.923742 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.924260 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.925482 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.925838 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.925909 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.926325 (+ 416us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:03:48.926399 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:03:48.926458 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:48.926595 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 03:03:48.926634 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 03:03:48.926813 18352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:48.922978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:39935 (local address 127.5.96.131:50645)
0812 03:03:48.923252 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:48.923265 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:48.923362 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:48.923537 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:48.923540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:48.923546 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:03:48.923632 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.923637 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.924368 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.924371 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.925343 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:48.925355 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.926043 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.926047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:48.926133 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:48.926179 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:03:48.926480 (+ 301us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:03:48.926486 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:03:48.926580 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 03:03:48.926658 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":95,"threads_started":1}
I20260812 03:03:48.948727 17407 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:48.948886 17407 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:48.950990 17407 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e5401c1b56aa4fe08ea1f21b4f3bfe88: Sending DeleteTablet for 3 replicas of tablet 98db608a9e1d4569bbda5c963cdbffcd
I20260812 03:03:48.951573 18090 tablet_service.cc:1514] Processing DeleteTablet for tablet 98db608a9e1d4569bbda5c963cdbffcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:48 UTC) from {username='oryx', principal='oryx/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:33962
I20260812 03:03:48.951602 17874 tablet_service.cc:1514] Processing DeleteTablet for tablet 98db608a9e1d4569bbda5c963cdbffcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:48 UTC) from {username='oryx', principal='oryx/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:41028
I20260812 03:03:48.951799 18292 tablet_replica.cc:330] T 98db608a9e1d4569bbda5c963cdbffcd P 744ee7761ce7492fb7667b74afe25d96: stopping tablet replica
I20260812 03:03:48.951924 18292 raft_consensus.cc:2229] T 98db608a9e1d4569bbda5c963cdbffcd P 744ee7761ce7492fb7667b74afe25d96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:48.952046 18292 raft_consensus.cc:2258] T 98db608a9e1d4569bbda5c963cdbffcd P 744ee7761ce7492fb7667b74afe25d96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:48.952117 17637 tablet_service.cc:1514] Processing DeleteTablet for tablet 98db608a9e1d4569bbda5c963cdbffcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:48 UTC) from {username='oryx', principal='oryx/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:36112
I20260812 03:03:48.952208 18291 tablet_replica.cc:330] T 98db608a9e1d4569bbda5c963cdbffcd P b4002aff86fb431d968dc658fe8d8aab: stopping tablet replica
I20260812 03:03:48.952316 18291 raft_consensus.cc:2229] T 98db608a9e1d4569bbda5c963cdbffcd P b4002aff86fb431d968dc658fe8d8aab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:48.952334 18293 tablet_replica.cc:330] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e: stopping tablet replica
I20260812 03:03:48.952420 18291 raft_consensus.cc:2258] T 98db608a9e1d4569bbda5c963cdbffcd P b4002aff86fb431d968dc658fe8d8aab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:48.952426 18293 raft_consensus.cc:2229] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:03:48.952668 18293 raft_consensus.cc:2258] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:48.953114 18292 ts_tablet_manager.cc:1905] T 98db608a9e1d4569bbda5c963cdbffcd P 744ee7761ce7492fb7667b74afe25d96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:48.953619 18291 ts_tablet_manager.cc:1905] T 98db608a9e1d4569bbda5c963cdbffcd P b4002aff86fb431d968dc658fe8d8aab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:48.955976 18292 ts_tablet_manager.cc:1918] T 98db608a9e1d4569bbda5c963cdbffcd P 744ee7761ce7492fb7667b74afe25d96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:03:48.956045 18292 log.cc:1196] T 98db608a9e1d4569bbda5c963cdbffcd P 744ee7761ce7492fb7667b74afe25d96: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-1/wal/wals/98db608a9e1d4569bbda5c963cdbffcd
I20260812 03:03:48.956264 18293 ts_tablet_manager.cc:1905] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:48.956302 18292 ts_tablet_manager.cc:1939] T 98db608a9e1d4569bbda5c963cdbffcd P 744ee7761ce7492fb7667b74afe25d96: Deleting consensus metadata
I20260812 03:03:48.958933 18293 ts_tablet_manager.cc:1918] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:03:48.959017 18293 log.cc:1196] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-0/wal/wals/98db608a9e1d4569bbda5c963cdbffcd
I20260812 03:03:48.959271 18293 ts_tablet_manager.cc:1939] T 98db608a9e1d4569bbda5c963cdbffcd P e907e57623154d839f025bac02fcc26e: Deleting consensus metadata
I20260812 03:03:48.959867 17390 catalog_manager.cc:4841] TS 744ee7761ce7492fb7667b74afe25d96 (127.5.96.130:33917): tablet 98db608a9e1d4569bbda5c963cdbffcd (table test-table [id=5322a9d315e54b4da6a8dd837c090b03]) successfully deleted
I20260812 03:03:48.960247 17394 catalog_manager.cc:4841] TS e907e57623154d839f025bac02fcc26e (127.5.96.129:39935): tablet 98db608a9e1d4569bbda5c963cdbffcd (table test-table [id=5322a9d315e54b4da6a8dd837c090b03]) successfully deleted
I20260812 03:03:48.964600 18291 ts_tablet_manager.cc:1918] T 98db608a9e1d4569bbda5c963cdbffcd P b4002aff86fb431d968dc658fe8d8aab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:03:48.964737 18291 log.cc:1196] T 98db608a9e1d4569bbda5c963cdbffcd P b4002aff86fb431d968dc658fe8d8aab: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-2/wal/wals/98db608a9e1d4569bbda5c963cdbffcd
I20260812 03:03:48.965031 18291 ts_tablet_manager.cc:1939] T 98db608a9e1d4569bbda5c963cdbffcd P b4002aff86fb431d968dc658fe8d8aab: Deleting consensus metadata
I20260812 03:03:48.965720 17392 catalog_manager.cc:4841] TS b4002aff86fb431d968dc658fe8d8aab (127.5.96.131:40781): tablet 98db608a9e1d4569bbda5c963cdbffcd (table test-table [id=5322a9d315e54b4da6a8dd837c090b03]) successfully deleted
Aug 12 03:03:48 dist-test-slave-6bbx krb5kdc[17338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:03:48 dist-test-slave-6bbx krb5kdc[17338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:48 dist-test-slave-6bbx krb5kdc[17338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 03:03:49.001524 17499 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:48.988253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47806 (local address 127.5.96.190:35965)
0812 03:03:48.988376 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:48.988380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:48.994212 (+ 5832us) server_negotiation.cc:407] Connection header received
0812 03:03:48.994429 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:48.994433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:48.994483 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:48.994531 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:48.995659 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.996483 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.997672 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:48.997972 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:48.998041 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:49.001403 (+ 3362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47806: BlockingRecv error: recv got EOF from 127.0.0.1:47806 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 12 03:03:49 dist-test-slave-6bbx krb5kdc[17338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM
I20260812 03:03:49.010381 17499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:49.002990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47812 (local address 127.5.96.190:35965)
0812 03:03:49.003089 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:49.003093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:49.003252 (+ 159us) server_negotiation.cc:407] Connection header received
0812 03:03:49.003354 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:49.003357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:49.003398 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:49.003461 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:49.004047 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:49.004522 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:49.005241 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:49.005457 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:49.005540 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:49.007636 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:49.007640 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:49.007643 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:49.007689 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:49.008934 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:49.009295 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:49.009299 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:49.009301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:49.009381 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:49.009621 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:49.009624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:49.009626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:49.009796 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:49.009924 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:49.010136 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 03:03:49.010239 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:03:49.012871 17407 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47812:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:03:49.013286 17407 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:03:49.017675 17637 tablet_service.cc:1467] Processing CreateTablet for tablet 8e38934fcdbd47bba5dc059f7187199b (DEFAULT_TABLE table=test-table [id=63879063485e49ff94ff843b20dffbff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:49.017731 17874 tablet_service.cc:1467] Processing CreateTablet for tablet 8e38934fcdbd47bba5dc059f7187199b (DEFAULT_TABLE table=test-table [id=63879063485e49ff94ff843b20dffbff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:49.017975 17637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e38934fcdbd47bba5dc059f7187199b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:49.017999 17874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e38934fcdbd47bba5dc059f7187199b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:49.017990 18090 tablet_service.cc:1467] Processing CreateTablet for tablet 8e38934fcdbd47bba5dc059f7187199b (DEFAULT_TABLE table=test-table [id=63879063485e49ff94ff843b20dffbff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:49.018229 18090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e38934fcdbd47bba5dc059f7187199b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:49.020414 18256 tablet_bootstrap.cc:492] T 8e38934fcdbd47bba5dc059f7187199b P e907e57623154d839f025bac02fcc26e: Bootstrap starting.
I20260812 03:03:49.020411 18250 tablet_bootstrap.cc:492] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96: Bootstrap starting.
I20260812 03:03:49.021116 18256 tablet_bootstrap.cc:654] T 8e38934fcdbd47bba5dc059f7187199b P e907e57623154d839f025bac02fcc26e: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:49.021189 18250 tablet_bootstrap.cc:654] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:49.021626 18252 tablet_bootstrap.cc:492] T 8e38934fcdbd47bba5dc059f7187199b P b4002aff86fb431d968dc658fe8d8aab: Bootstrap starting.
I20260812 03:03:49.022324 18250 tablet_bootstrap.cc:492] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96: No bootstrap required, opened a new log
I20260812 03:03:49.022396 18250 ts_tablet_manager.cc:1397] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:03:49.022416 18252 tablet_bootstrap.cc:654] T 8e38934fcdbd47bba5dc059f7187199b P b4002aff86fb431d968dc658fe8d8aab: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:49.022748 18250 raft_consensus.cc:348] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:49.022845 18250 raft_consensus.cc:374] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:49.022874 18250 raft_consensus.cc:729] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 744ee7761ce7492fb7667b74afe25d96, State: Initialized, Role: FOLLOWER
I20260812 03:03:49.022984 18250 consensus_queue.cc:260] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:49.023217 18250 ts_tablet_manager.cc:1428] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:03:49.023870 18256 tablet_bootstrap.cc:492] T 8e38934fcdbd47bba5dc059f7187199b P e907e57623154d839f025bac02fcc26e: No bootstrap required, opened a new log
I20260812 03:03:49.023957 18256 ts_tablet_manager.cc:1397] T 8e38934fcdbd47bba5dc059f7187199b P e907e57623154d839f025bac02fcc26e: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260812 03:03:49.024358 18256 raft_consensus.cc:348] T 8e38934fcdbd47bba5dc059f7187199b P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:49.024482 18256 raft_consensus.cc:374] T 8e38934fcdbd47bba5dc059f7187199b P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:49.024510 18256 raft_consensus.cc:729] T 8e38934fcdbd47bba5dc059f7187199b P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e907e57623154d839f025bac02fcc26e, State: Initialized, Role: FOLLOWER
I20260812 03:03:49.024629 18256 consensus_queue.cc:260] T 8e38934fcdbd47bba5dc059f7187199b P e907e57623154d839f025bac02fcc26e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:49.024879 18256 ts_tablet_manager.cc:1428] T 8e38934fcdbd47bba5dc059f7187199b P e907e57623154d839f025bac02fcc26e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:03:49.025763 18252 tablet_bootstrap.cc:492] T 8e38934fcdbd47bba5dc059f7187199b P b4002aff86fb431d968dc658fe8d8aab: No bootstrap required, opened a new log
I20260812 03:03:49.025859 18252 ts_tablet_manager.cc:1397] T 8e38934fcdbd47bba5dc059f7187199b P b4002aff86fb431d968dc658fe8d8aab: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 03:03:49.026188 18252 raft_consensus.cc:348] T 8e38934fcdbd47bba5dc059f7187199b P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:49.026293 18252 raft_consensus.cc:374] T 8e38934fcdbd47bba5dc059f7187199b P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:49.026327 18252 raft_consensus.cc:729] T 8e38934fcdbd47bba5dc059f7187199b P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4002aff86fb431d968dc658fe8d8aab, State: Initialized, Role: FOLLOWER
I20260812 03:03:49.026439 18252 consensus_queue.cc:260] T 8e38934fcdbd47bba5dc059f7187199b P b4002aff86fb431d968dc658fe8d8aab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:49.026690 18252 ts_tablet_manager.cc:1428] T 8e38934fcdbd47bba5dc059f7187199b P b4002aff86fb431d968dc658fe8d8aab: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:03:49.131194 18287 raft_consensus.cc:482] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:03:49.131320 18287 raft_consensus.cc:504] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:49.131696 18287 leader_election.cc:288] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e907e57623154d839f025bac02fcc26e (127.5.96.129:39935), b4002aff86fb431d968dc658fe8d8aab (127.5.96.131:40781)
I20260812 03:03:49.132085 17673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e38934fcdbd47bba5dc059f7187199b" candidate_uuid: "744ee7761ce7492fb7667b74afe25d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e907e57623154d839f025bac02fcc26e" is_pre_election: true
I20260812 03:03:49.132093 18122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e38934fcdbd47bba5dc059f7187199b" candidate_uuid: "744ee7761ce7492fb7667b74afe25d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4002aff86fb431d968dc658fe8d8aab" is_pre_election: true
I20260812 03:03:49.132220 18122 raft_consensus.cc:2454] T 8e38934fcdbd47bba5dc059f7187199b P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 744ee7761ce7492fb7667b74afe25d96 in term 0.
I20260812 03:03:49.132239 17673 raft_consensus.cc:2454] T 8e38934fcdbd47bba5dc059f7187199b P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 744ee7761ce7492fb7667b74afe25d96 in term 0.
I20260812 03:03:49.132493 17792 leader_election.cc:302] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 744ee7761ce7492fb7667b74afe25d96, b4002aff86fb431d968dc658fe8d8aab; no voters:
I20260812 03:03:49.132663 18287 raft_consensus.cc:2781] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:03:49.132731 18287 raft_consensus.cc:482] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:03:49.132774 18287 raft_consensus.cc:3037] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:49.133744 18287 raft_consensus.cc:504] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:49.134019 18287 leader_election.cc:288] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96 [CANDIDATE]: Term 1 election: Requested vote from peers e907e57623154d839f025bac02fcc26e (127.5.96.129:39935), b4002aff86fb431d968dc658fe8d8aab (127.5.96.131:40781)
I20260812 03:03:49.134338 17673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e38934fcdbd47bba5dc059f7187199b" candidate_uuid: "744ee7761ce7492fb7667b74afe25d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e907e57623154d839f025bac02fcc26e"
I20260812 03:03:49.134359 18122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e38934fcdbd47bba5dc059f7187199b" candidate_uuid: "744ee7761ce7492fb7667b74afe25d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4002aff86fb431d968dc658fe8d8aab"
I20260812 03:03:49.134431 17673 raft_consensus.cc:3037] T 8e38934fcdbd47bba5dc059f7187199b P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:49.134464 18122 raft_consensus.cc:3037] T 8e38934fcdbd47bba5dc059f7187199b P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:49.135273 17673 raft_consensus.cc:2454] T 8e38934fcdbd47bba5dc059f7187199b P e907e57623154d839f025bac02fcc26e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 744ee7761ce7492fb7667b74afe25d96 in term 1.
I20260812 03:03:49.135360 18122 raft_consensus.cc:2454] T 8e38934fcdbd47bba5dc059f7187199b P b4002aff86fb431d968dc658fe8d8aab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 744ee7761ce7492fb7667b74afe25d96 in term 1.
I20260812 03:03:49.135512 17795 leader_election.cc:302] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 744ee7761ce7492fb7667b74afe25d96, e907e57623154d839f025bac02fcc26e; no voters:
I20260812 03:03:49.135694 18287 raft_consensus.cc:2781] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:49.135862 18287 raft_consensus.cc:686] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96 [term 1 LEADER]: Becoming Leader. State: Replica: 744ee7761ce7492fb7667b74afe25d96, State: Running, Role: LEADER
I20260812 03:03:49.136010 18287 consensus_queue.cc:237] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:49.137477 17407 catalog_manager.cc:5495] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 744ee7761ce7492fb7667b74afe25d96 (127.5.96.130). New cstate: current_term: 1 leader_uuid: "744ee7761ce7492fb7667b74afe25d96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } health_report { overall_health: UNKNOWN } } }
I20260812 03:03:49.151144 18247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:49.148166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41044 (local address 127.5.96.130:33917)
0812 03:03:49.148275 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:49.148279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:49.148356 (+ 77us) server_negotiation.cc:407] Connection header received
0812 03:03:49.148483 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:49.148487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:49.148531 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:49.148602 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:03:49.148834 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:49.149365 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:49.150187 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:49.150416 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:49.150502 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:49.150744 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:03:49.150822 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:03:49.150880 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:49.150970 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 03:03:49.151024 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:03:49.152733 18122 raft_consensus.cc:1261] T 8e38934fcdbd47bba5dc059f7187199b P b4002aff86fb431d968dc658fe8d8aab [term 1 FOLLOWER]: Refusing update from remote peer 744ee7761ce7492fb7667b74afe25d96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:49.152822 17673 raft_consensus.cc:1261] T 8e38934fcdbd47bba5dc059f7187199b P e907e57623154d839f025bac02fcc26e [term 1 FOLLOWER]: Refusing update from remote peer 744ee7761ce7492fb7667b74afe25d96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:49.153092 18287 consensus_queue.cc:1035] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:49.153213 18387 consensus_queue.cc:1035] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:49.159344 17409 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47812:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:49.159480 17409 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47812:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:49.161664 17409 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e5401c1b56aa4fe08ea1f21b4f3bfe88: Sending DeleteTablet for 3 replicas of tablet 8e38934fcdbd47bba5dc059f7187199b
I20260812 03:03:49.162192 17874 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e38934fcdbd47bba5dc059f7187199b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:49 UTC) from {username='oryx', principal='oryx/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:41028
I20260812 03:03:49.162216 17637 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e38934fcdbd47bba5dc059f7187199b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:49 UTC) from {username='oryx', principal='oryx/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:36112
I20260812 03:03:49.162287 17636 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e38934fcdbd47bba5dc059f7187199b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:49 UTC) from {username='oryx', principal='oryx/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:36112
I20260812 03:03:49.162366 18292 tablet_replica.cc:330] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96: stopping tablet replica
I20260812 03:03:49.162412 18293 tablet_replica.cc:330] T 8e38934fcdbd47bba5dc059f7187199b P e907e57623154d839f025bac02fcc26e: stopping tablet replica
I20260812 03:03:49.162487 18292 raft_consensus.cc:2229] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:03:49.162523 18293 raft_consensus.cc:2229] T 8e38934fcdbd47bba5dc059f7187199b P e907e57623154d839f025bac02fcc26e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:49.162643 18293 raft_consensus.cc:2258] T 8e38934fcdbd47bba5dc059f7187199b P e907e57623154d839f025bac02fcc26e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:49.162643 18292 raft_consensus.cc:2258] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:49.163039 18090 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e38934fcdbd47bba5dc059f7187199b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:49 UTC) from {username='oryx', principal='oryx/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:33962
I20260812 03:03:49.163198 18088 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e38934fcdbd47bba5dc059f7187199b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:49 UTC) from {username='oryx', principal='oryx/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:33962
I20260812 03:03:49.163326 18291 tablet_replica.cc:330] T 8e38934fcdbd47bba5dc059f7187199b P b4002aff86fb431d968dc658fe8d8aab: stopping tablet replica
I20260812 03:03:49.163311 18292 ts_tablet_manager.cc:1905] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:49.163324 17407 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47812:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 03:03:49.163420 18291 raft_consensus.cc:2229] T 8e38934fcdbd47bba5dc059f7187199b P b4002aff86fb431d968dc658fe8d8aab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:49.163446 18293 ts_tablet_manager.cc:1905] T 8e38934fcdbd47bba5dc059f7187199b P e907e57623154d839f025bac02fcc26e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:49.163530 18291 raft_consensus.cc:2258] T 8e38934fcdbd47bba5dc059f7187199b P b4002aff86fb431d968dc658fe8d8aab [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 03:03:49.163707 17407 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:03:49.164220 18291 ts_tablet_manager.cc:1905] T 8e38934fcdbd47bba5dc059f7187199b P b4002aff86fb431d968dc658fe8d8aab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:49.165503 18292 ts_tablet_manager.cc:1918] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:03:49.165556 18292 log.cc:1196] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-1/wal/wals/8e38934fcdbd47bba5dc059f7187199b
I20260812 03:03:49.165741 18292 ts_tablet_manager.cc:1939] T 8e38934fcdbd47bba5dc059f7187199b P 744ee7761ce7492fb7667b74afe25d96: Deleting consensus metadata
I20260812 03:03:49.165793 18293 ts_tablet_manager.cc:1918] T 8e38934fcdbd47bba5dc059f7187199b P e907e57623154d839f025bac02fcc26e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:03:49.165855 18293 log.cc:1196] T 8e38934fcdbd47bba5dc059f7187199b P e907e57623154d839f025bac02fcc26e: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-0/wal/wals/8e38934fcdbd47bba5dc059f7187199b
I20260812 03:03:49.166067 18293 ts_tablet_manager.cc:1939] T 8e38934fcdbd47bba5dc059f7187199b P e907e57623154d839f025bac02fcc26e: Deleting consensus metadata
I20260812 03:03:49.166321 17390 catalog_manager.cc:4841] TS 744ee7761ce7492fb7667b74afe25d96 (127.5.96.130:33917): tablet 8e38934fcdbd47bba5dc059f7187199b (table test-table [id=63879063485e49ff94ff843b20dffbff]) successfully deleted
I20260812 03:03:49.166529 18291 ts_tablet_manager.cc:1918] T 8e38934fcdbd47bba5dc059f7187199b P b4002aff86fb431d968dc658fe8d8aab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:03:49.166603 18291 log.cc:1196] T 8e38934fcdbd47bba5dc059f7187199b P b4002aff86fb431d968dc658fe8d8aab: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-2/wal/wals/8e38934fcdbd47bba5dc059f7187199b
I20260812 03:03:49.166685 17394 catalog_manager.cc:4841] TS e907e57623154d839f025bac02fcc26e (127.5.96.129:39935): tablet 8e38934fcdbd47bba5dc059f7187199b (table test-table [id=63879063485e49ff94ff843b20dffbff]) successfully deleted
I20260812 03:03:49.166831 18291 ts_tablet_manager.cc:1939] T 8e38934fcdbd47bba5dc059f7187199b P b4002aff86fb431d968dc658fe8d8aab: Deleting consensus metadata
W20260812 03:03:49.167006 17394 catalog_manager.cc:4808] TS e907e57623154d839f025bac02fcc26e (127.5.96.129:39935): delete failed for tablet 8e38934fcdbd47bba5dc059f7187199b because the tablet was not found. No further retry: Not found: Tablet not found: 8e38934fcdbd47bba5dc059f7187199b
I20260812 03:03:49.167405 17392 catalog_manager.cc:4841] TS b4002aff86fb431d968dc658fe8d8aab (127.5.96.131:40781): tablet 8e38934fcdbd47bba5dc059f7187199b (table test-table [id=63879063485e49ff94ff843b20dffbff]) successfully deleted
W20260812 03:03:49.167559 17392 catalog_manager.cc:4808] TS b4002aff86fb431d968dc658fe8d8aab (127.5.96.131:40781): delete failed for tablet 8e38934fcdbd47bba5dc059f7187199b because the tablet was not found. No further retry: Not found: Tablet not found: 8e38934fcdbd47bba5dc059f7187199b
I20260812 03:03:49.170856 17636 tablet_service.cc:1467] Processing CreateTablet for tablet 9f97bd89eb36481da84ddcb26826f0ba (DEFAULT_TABLE table=test-table [id=62cd9401a7bc4c9fa7a7298adfd8b1cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:49.170882 17874 tablet_service.cc:1467] Processing CreateTablet for tablet 9f97bd89eb36481da84ddcb26826f0ba (DEFAULT_TABLE table=test-table [id=62cd9401a7bc4c9fa7a7298adfd8b1cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:49.170923 18088 tablet_service.cc:1467] Processing CreateTablet for tablet 9f97bd89eb36481da84ddcb26826f0ba (DEFAULT_TABLE table=test-table [id=62cd9401a7bc4c9fa7a7298adfd8b1cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:03:49.171115 17636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f97bd89eb36481da84ddcb26826f0ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:49.171119 17874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f97bd89eb36481da84ddcb26826f0ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:49.171144 18088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f97bd89eb36481da84ddcb26826f0ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:49.173058 18256 tablet_bootstrap.cc:492] T 9f97bd89eb36481da84ddcb26826f0ba P e907e57623154d839f025bac02fcc26e: Bootstrap starting.
I20260812 03:03:49.173249 18250 tablet_bootstrap.cc:492] T 9f97bd89eb36481da84ddcb26826f0ba P 744ee7761ce7492fb7667b74afe25d96: Bootstrap starting.
I20260812 03:03:49.173269 18252 tablet_bootstrap.cc:492] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab: Bootstrap starting.
I20260812 03:03:49.173869 18256 tablet_bootstrap.cc:654] T 9f97bd89eb36481da84ddcb26826f0ba P e907e57623154d839f025bac02fcc26e: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:49.173977 18252 tablet_bootstrap.cc:654] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:49.173977 18250 tablet_bootstrap.cc:654] T 9f97bd89eb36481da84ddcb26826f0ba P 744ee7761ce7492fb7667b74afe25d96: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:49.174993 18250 tablet_bootstrap.cc:492] T 9f97bd89eb36481da84ddcb26826f0ba P 744ee7761ce7492fb7667b74afe25d96: No bootstrap required, opened a new log
I20260812 03:03:49.175061 18250 ts_tablet_manager.cc:1397] T 9f97bd89eb36481da84ddcb26826f0ba P 744ee7761ce7492fb7667b74afe25d96: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:03:49.175181 18256 tablet_bootstrap.cc:492] T 9f97bd89eb36481da84ddcb26826f0ba P e907e57623154d839f025bac02fcc26e: No bootstrap required, opened a new log
I20260812 03:03:49.175261 18256 ts_tablet_manager.cc:1397] T 9f97bd89eb36481da84ddcb26826f0ba P e907e57623154d839f025bac02fcc26e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:03:49.175439 18250 raft_consensus.cc:348] T 9f97bd89eb36481da84ddcb26826f0ba P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:49.175547 18250 raft_consensus.cc:374] T 9f97bd89eb36481da84ddcb26826f0ba P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:49.175588 18250 raft_consensus.cc:729] T 9f97bd89eb36481da84ddcb26826f0ba P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 744ee7761ce7492fb7667b74afe25d96, State: Initialized, Role: FOLLOWER
I20260812 03:03:49.175570 18256 raft_consensus.cc:348] T 9f97bd89eb36481da84ddcb26826f0ba P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:49.175702 18256 raft_consensus.cc:374] T 9f97bd89eb36481da84ddcb26826f0ba P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:49.175755 18256 raft_consensus.cc:729] T 9f97bd89eb36481da84ddcb26826f0ba P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e907e57623154d839f025bac02fcc26e, State: Initialized, Role: FOLLOWER
I20260812 03:03:49.175721 18250 consensus_queue.cc:260] T 9f97bd89eb36481da84ddcb26826f0ba P 744ee7761ce7492fb7667b74afe25d96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:49.175876 18256 consensus_queue.cc:260] T 9f97bd89eb36481da84ddcb26826f0ba P e907e57623154d839f025bac02fcc26e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:49.175978 18250 ts_tablet_manager.cc:1428] T 9f97bd89eb36481da84ddcb26826f0ba P 744ee7761ce7492fb7667b74afe25d96: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:03:49.176138 18256 ts_tablet_manager.cc:1428] T 9f97bd89eb36481da84ddcb26826f0ba P e907e57623154d839f025bac02fcc26e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:03:49.176924 18252 tablet_bootstrap.cc:492] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab: No bootstrap required, opened a new log
I20260812 03:03:49.176997 18252 ts_tablet_manager.cc:1397] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 03:03:49.177318 18252 raft_consensus.cc:348] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:49.177418 18252 raft_consensus.cc:374] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:49.177449 18252 raft_consensus.cc:729] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4002aff86fb431d968dc658fe8d8aab, State: Initialized, Role: FOLLOWER
I20260812 03:03:49.177556 18252 consensus_queue.cc:260] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:49.177804 18252 ts_tablet_manager.cc:1428] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:03:49.189174 18266 raft_consensus.cc:482] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:03:49.189261 18266 raft_consensus.cc:504] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:49.189571 18266 leader_election.cc:288] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e907e57623154d839f025bac02fcc26e (127.5.96.129:39935), 744ee7761ce7492fb7667b74afe25d96 (127.5.96.130:33917)
I20260812 03:03:49.189863 17906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f97bd89eb36481da84ddcb26826f0ba" candidate_uuid: "b4002aff86fb431d968dc658fe8d8aab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "744ee7761ce7492fb7667b74afe25d96" is_pre_election: true
I20260812 03:03:49.189863 17673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f97bd89eb36481da84ddcb26826f0ba" candidate_uuid: "b4002aff86fb431d968dc658fe8d8aab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e907e57623154d839f025bac02fcc26e" is_pre_election: true
I20260812 03:03:49.189973 17906 raft_consensus.cc:2454] T 9f97bd89eb36481da84ddcb26826f0ba P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4002aff86fb431d968dc658fe8d8aab in term 0.
I20260812 03:03:49.189996 17673 raft_consensus.cc:2454] T 9f97bd89eb36481da84ddcb26826f0ba P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4002aff86fb431d968dc658fe8d8aab in term 0.
I20260812 03:03:49.190210 18016 leader_election.cc:302] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 744ee7761ce7492fb7667b74afe25d96, b4002aff86fb431d968dc658fe8d8aab; no voters:
I20260812 03:03:49.190402 18266 raft_consensus.cc:2781] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:03:49.190467 18266 raft_consensus.cc:482] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:03:49.190505 18266 raft_consensus.cc:3037] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:49.191396 18266 raft_consensus.cc:504] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:49.191773 18266 leader_election.cc:288] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab [CANDIDATE]: Term 1 election: Requested vote from peers e907e57623154d839f025bac02fcc26e (127.5.96.129:39935), 744ee7761ce7492fb7667b74afe25d96 (127.5.96.130:33917)
I20260812 03:03:49.192030 17906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f97bd89eb36481da84ddcb26826f0ba" candidate_uuid: "b4002aff86fb431d968dc658fe8d8aab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "744ee7761ce7492fb7667b74afe25d96"
I20260812 03:03:49.192034 17673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f97bd89eb36481da84ddcb26826f0ba" candidate_uuid: "b4002aff86fb431d968dc658fe8d8aab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e907e57623154d839f025bac02fcc26e"
I20260812 03:03:49.192123 17673 raft_consensus.cc:3037] T 9f97bd89eb36481da84ddcb26826f0ba P e907e57623154d839f025bac02fcc26e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:49.192126 17906 raft_consensus.cc:3037] T 9f97bd89eb36481da84ddcb26826f0ba P 744ee7761ce7492fb7667b74afe25d96 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:49.192854 17906 raft_consensus.cc:2454] T 9f97bd89eb36481da84ddcb26826f0ba P 744ee7761ce7492fb7667b74afe25d96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4002aff86fb431d968dc658fe8d8aab in term 1.
I20260812 03:03:49.193034 17673 raft_consensus.cc:2454] T 9f97bd89eb36481da84ddcb26826f0ba P e907e57623154d839f025bac02fcc26e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4002aff86fb431d968dc658fe8d8aab in term 1.
I20260812 03:03:49.193107 18016 leader_election.cc:302] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 744ee7761ce7492fb7667b74afe25d96, b4002aff86fb431d968dc658fe8d8aab; no voters:
I20260812 03:03:49.193375 18266 raft_consensus.cc:2781] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:49.193461 18266 raft_consensus.cc:686] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab [term 1 LEADER]: Becoming Leader. State: Replica: b4002aff86fb431d968dc658fe8d8aab, State: Running, Role: LEADER
I20260812 03:03:49.193564 18266 consensus_queue.cc:237] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } }
I20260812 03:03:49.194962 17407 catalog_manager.cc:5495] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab reported cstate change: term changed from 0 to 1, leader changed from <none> to b4002aff86fb431d968dc658fe8d8aab (127.5.96.131). New cstate: current_term: 1 leader_uuid: "b4002aff86fb431d968dc658fe8d8aab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4002aff86fb431d968dc658fe8d8aab" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 40781 } health_report { overall_health: HEALTHY } } }
I20260812 03:03:49.213615 18242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:49.210663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33992 (local address 127.5.96.131:40781)
0812 03:03:49.210781 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:49.210785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:49.210798 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:03:49.210892 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:49.210895 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:49.210936 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:49.210992 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:03:49.211201 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:49.211757 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:49.212559 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:49.212897 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:49.213008 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:49.213237 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:03:49.213304 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:03:49.213360 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:49.213455 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 03:03:49.213505 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 03:03:49.220944 18242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:49.218400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:37875 (local address 127.5.96.131:40781)
0812 03:03:49.218487 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:49.218491 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:49.218503 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:03:49.218592 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:49.218595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:49.218631 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:49.218684 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:03:49.218861 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:49.219302 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:49.220095 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:49.220296 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:49.220376 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:49.220588 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:03:49.220643 (+ 55us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:03:49.220698 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:49.220783 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 03:03:49.220823 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":34}
I20260812 03:03:49.220953 18352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:49.218346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:40781 (local address 127.5.96.131:37875)
0812 03:03:49.218442 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:49.218455 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:49.218525 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:49.218697 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:49.218700 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:49.218705 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:03:49.218778 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:49.218783 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:49.219391 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:49.219394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:49.219997 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:49.220006 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:49.220384 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:49.220388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:49.220476 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:49.220497 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:03:49.220715 (+ 218us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:03:49.220721 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:03:49.220777 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 03:03:49.220854 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":1}
I20260812 03:03:49.225451 17673 raft_consensus.cc:1261] T 9f97bd89eb36481da84ddcb26826f0ba P e907e57623154d839f025bac02fcc26e [term 1 FOLLOWER]: Refusing update from remote peer b4002aff86fb431d968dc658fe8d8aab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:49.225566 17906 raft_consensus.cc:1261] T 9f97bd89eb36481da84ddcb26826f0ba P 744ee7761ce7492fb7667b74afe25d96 [term 1 FOLLOWER]: Refusing update from remote peer b4002aff86fb431d968dc658fe8d8aab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:03:49.225790 18345 consensus_queue.cc:1035] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e907e57623154d839f025bac02fcc26e" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:49.225921 18266 consensus_queue.cc:1035] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab [LEADER]: Connected to new peer: Peer: permanent_uuid: "744ee7761ce7492fb7667b74afe25d96" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:03:49.254673 17409 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47812:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:49.254798 17409 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47812:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:03:49.256500 17409 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e5401c1b56aa4fe08ea1f21b4f3bfe88: Sending DeleteTablet for 3 replicas of tablet 9f97bd89eb36481da84ddcb26826f0ba
I20260812 03:03:49.257002 17636 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f97bd89eb36481da84ddcb26826f0ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:49 UTC) from {username='oryx', principal='oryx/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:36112
I20260812 03:03:49.257052 18088 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f97bd89eb36481da84ddcb26826f0ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:49 UTC) from {username='oryx', principal='oryx/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:33962
I20260812 03:03:49.257051 17874 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f97bd89eb36481da84ddcb26826f0ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:03:49 UTC) from {username='oryx', principal='oryx/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:41028
I20260812 03:03:49.257167 18293 tablet_replica.cc:330] T 9f97bd89eb36481da84ddcb26826f0ba P e907e57623154d839f025bac02fcc26e: stopping tablet replica
I20260812 03:03:49.257195 18291 tablet_replica.cc:330] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab: stopping tablet replica
I20260812 03:03:49.257195 18292 tablet_replica.cc:330] T 9f97bd89eb36481da84ddcb26826f0ba P 744ee7761ce7492fb7667b74afe25d96: stopping tablet replica
I20260812 03:03:49.257283 18293 raft_consensus.cc:2229] T 9f97bd89eb36481da84ddcb26826f0ba P e907e57623154d839f025bac02fcc26e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:49.257289 18291 raft_consensus.cc:2229] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:03:49.257300 18292 raft_consensus.cc:2229] T 9f97bd89eb36481da84ddcb26826f0ba P 744ee7761ce7492fb7667b74afe25d96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:03:49.257375 18293 raft_consensus.cc:2258] T 9f97bd89eb36481da84ddcb26826f0ba P e907e57623154d839f025bac02fcc26e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:49.257382 18292 raft_consensus.cc:2258] T 9f97bd89eb36481da84ddcb26826f0ba P 744ee7761ce7492fb7667b74afe25d96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:49.257536 18291 raft_consensus.cc:2258] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:03:49.258018 18293 ts_tablet_manager.cc:1905] T 9f97bd89eb36481da84ddcb26826f0ba P e907e57623154d839f025bac02fcc26e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:49.258214 18291 ts_tablet_manager.cc:1905] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:49.258214 18292 ts_tablet_manager.cc:1905] T 9f97bd89eb36481da84ddcb26826f0ba P 744ee7761ce7492fb7667b74afe25d96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:03:49.260452 18293 ts_tablet_manager.cc:1918] T 9f97bd89eb36481da84ddcb26826f0ba P e907e57623154d839f025bac02fcc26e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:03:49.260461 18291 ts_tablet_manager.cc:1918] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:03:49.260465 18292 ts_tablet_manager.cc:1918] T 9f97bd89eb36481da84ddcb26826f0ba P 744ee7761ce7492fb7667b74afe25d96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:03:49.260515 18293 log.cc:1196] T 9f97bd89eb36481da84ddcb26826f0ba P e907e57623154d839f025bac02fcc26e: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-0/wal/wals/9f97bd89eb36481da84ddcb26826f0ba
I20260812 03:03:49.260529 18291 log.cc:1196] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-2/wal/wals/9f97bd89eb36481da84ddcb26826f0ba
I20260812 03:03:49.260529 18292 log.cc:1196] T 9f97bd89eb36481da84ddcb26826f0ba P 744ee7761ce7492fb7667b74afe25d96: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786503790637001-5506-0/minicluster-data/ts-1/wal/wals/9f97bd89eb36481da84ddcb26826f0ba
I20260812 03:03:49.260751 18293 ts_tablet_manager.cc:1939] T 9f97bd89eb36481da84ddcb26826f0ba P e907e57623154d839f025bac02fcc26e: Deleting consensus metadata
I20260812 03:03:49.260776 18292 ts_tablet_manager.cc:1939] T 9f97bd89eb36481da84ddcb26826f0ba P 744ee7761ce7492fb7667b74afe25d96: Deleting consensus metadata
I20260812 03:03:49.260802 18291 ts_tablet_manager.cc:1939] T 9f97bd89eb36481da84ddcb26826f0ba P b4002aff86fb431d968dc658fe8d8aab: Deleting consensus metadata
I20260812 03:03:49.261363 17394 catalog_manager.cc:4841] TS e907e57623154d839f025bac02fcc26e (127.5.96.129:39935): tablet 9f97bd89eb36481da84ddcb26826f0ba (table test-table [id=62cd9401a7bc4c9fa7a7298adfd8b1cf]) successfully deleted
I20260812 03:03:49.261557 17390 catalog_manager.cc:4841] TS 744ee7761ce7492fb7667b74afe25d96 (127.5.96.130:33917): tablet 9f97bd89eb36481da84ddcb26826f0ba (table test-table [id=62cd9401a7bc4c9fa7a7298adfd8b1cf]) successfully deleted
I20260812 03:03:49.261909 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 17536
I20260812 03:03:49.269933 17392 catalog_manager.cc:4841] TS b4002aff86fb431d968dc658fe8d8aab (127.5.96.131:40781): tablet 9f97bd89eb36481da84ddcb26826f0ba (table test-table [id=62cd9401a7bc4c9fa7a7298adfd8b1cf]) successfully deleted
I20260812 03:03:49.278604 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 17768
I20260812 03:03:49.297044 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 17990
I20260812 03:03:49.307885 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 17365
2026-08-12T03:03:49Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3240 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:03:49 dist-test-slave-6bbx krb5kdc[18416](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:03:49 dist-test-slave-6bbx krb5kdc[18416](info): set up 2 sockets
Aug 12 03:03:49 dist-test-slave-6bbx krb5kdc[18416](info): commencing operation
krb5kdc: starting...
W20260812 03:03:50.465138 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:03:50 dist-test-slave-6bbx krb5kdc[18416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:03:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:03:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:03:50.574957 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:39967
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:43251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:39967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:50.681434 18435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:50.681674 18435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:50.681727 18435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:50.685585 18435 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:03:50.685634 18435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:50.685660 18435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:50.685679 18435 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:03:50.685699 18435 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:03:50.691102 18435 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:43251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:39967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:39967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:50.692073 18435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:50.693053 18435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:50.701045 18446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:50.701066 18450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:50.701081 18435 server_base.cc:1034] running on GCE node
W20260812 03:03:50.701045 18447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:50.701807 18435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:50.702818 18435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:50.703987 18435 hybrid_clock.cc:648] HybridClock initialized: now 1786503830703965 us; error 31 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T03:03:50Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1398 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:03:50 dist-test-slave-6bbx krb5kdc[18455](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:03:50 dist-test-slave-6bbx krb5kdc[18455](info): set up 2 sockets
Aug 12 03:03:50 dist-test-slave-6bbx krb5kdc[18455](info): commencing operation
krb5kdc: starting...
W20260812 03:03:52.059429 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.326s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:03:52 dist-test-slave-6bbx krb5kdc[18455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:03:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:03:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:03:52.173827 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:37713
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:34069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:37713
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:52.282389 18476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:52.282614 18476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:52.282655 18476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:52.286521 18476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:03:52.286567 18476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:52.286587 18476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:52.286616 18476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:03:52.286644 18476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:03:52.291774 18476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:34069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:37713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:37713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:52.292706 18476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:52.293627 18476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:52.301564 18485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:52.301586 18486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:52.301563 18491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:52.301846 18476 server_base.cc:1034] running on GCE node
I20260812 03:03:52.302462 18476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:52.303448 18476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:52.304616 18476 hybrid_clock.cc:648] HybridClock initialized: now 1786503832304596 us; error 36 us; skew 500 ppm
Aug 12 03:03:52 dist-test-slave-6bbx krb5kdc[18455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503832, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:52.309894 18476 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:03:52.310740 18476 webserver.cc:460] Webserver started at http://127.5.96.190:33737/ using document root <none> and password file <none>
I20260812 03:03:52.311268 18476 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:52.311335 18476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:52.311532 18476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:52.313499 18476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "4a201e692d0641b69c6215281d41f0c8"
format_stamp: "Formatted at 2026-08-12 03:03:52 on dist-test-slave-6bbx"
server_key: "ffcaa4219a2c163e5ad3e8eb846b7c70"
server_key_iv: "f076dad2641ec5b06fbda9628f25b99b"
server_key_version: "encryptionkey@0"
I20260812 03:03:52.314034 18476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "4a201e692d0641b69c6215281d41f0c8"
format_stamp: "Formatted at 2026-08-12 03:03:52 on dist-test-slave-6bbx"
server_key: "ffcaa4219a2c163e5ad3e8eb846b7c70"
server_key_iv: "f076dad2641ec5b06fbda9628f25b99b"
server_key_version: "encryptionkey@0"
I20260812 03:03:52.317589 18476 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 03:03:52.319806 18503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:52.320946 18476 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:52.321071 18476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "4a201e692d0641b69c6215281d41f0c8"
format_stamp: "Formatted at 2026-08-12 03:03:52 on dist-test-slave-6bbx"
server_key: "ffcaa4219a2c163e5ad3e8eb846b7c70"
server_key_iv: "f076dad2641ec5b06fbda9628f25b99b"
server_key_version: "encryptionkey@0"
I20260812 03:03:52.321180 18476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:52.338004 18476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:52.340531 18476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:52.340698 18476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:52.347407 18476 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:37713
I20260812 03:03:52.347448 18593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:37713 every 8 connection(s)
I20260812 03:03:52.348307 18476 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:03:52.349128 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 18476
I20260812 03:03:52.349251 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:03:52.349496 5506 external_mini_cluster.cc:1428] Setting key d5e08e0bb0063c1470f9c2c1ae41565a
I20260812 03:03:52.351135 18594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:03:52.357573 18594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8: Bootstrap starting.
Aug 12 03:03:52 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:52.360087 18594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8: Neither blocks nor log segments found. Creating new log.
I20260812 03:03:52.360942 18594 log.cc:824] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:52.363061 18594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8: No bootstrap required, opened a new log
I20260812 03:03:52.363806 18597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:52.350777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56664 (local address 127.5.96.190:37713)
0812 03:03:52.351191 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:52.351211 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:52.351251 (+ 40us) server_negotiation.cc:407] Connection header received
0812 03:03:52.352079 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:52.352102 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:52.352482 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:52.352833 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:52.353432 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:52.354887 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:52.355720 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:52.356382 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:52.356540 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:52.358954 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:52.358960 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:52.358974 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:52.359003 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:52.360777 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:52.361233 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:52.361239 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:52.361245 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:52.361370 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:52.361691 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:52.361695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:52.361696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:52.362186 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:52.362360 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:52.362571 (+ 211us) server_negotiation.cc:299] Negotiation successful
0812 03:03:52.362858 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":120,"threads_started":1}
I20260812 03:03:52.366950 18594 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a201e692d0641b69c6215281d41f0c8" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37713 } }
I20260812 03:03:52.367147 18594 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:03:52.367194 18594 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a201e692d0641b69c6215281d41f0c8, State: Initialized, Role: FOLLOWER
I20260812 03:03:52.367568 18594 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a201e692d0641b69c6215281d41f0c8" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37713 } }
I20260812 03:03:52.367712 18594 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:03:52.367767 18594 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:03:52.367838 18594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:03:52.368611 18594 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a201e692d0641b69c6215281d41f0c8" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37713 } }
I20260812 03:03:52.368892 18594 leader_election.cc:302] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a201e692d0641b69c6215281d41f0c8; no voters:
I20260812 03:03:52.369163 18594 leader_election.cc:288] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:03:52.369293 18599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:03:52.369537 18599 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [term 1 LEADER]: Becoming Leader. State: Replica: 4a201e692d0641b69c6215281d41f0c8, State: Running, Role: LEADER
I20260812 03:03:52.369813 18599 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a201e692d0641b69c6215281d41f0c8" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37713 } }
I20260812 03:03:52.370224 18594 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:03:52.371286 18604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a201e692d0641b69c6215281d41f0c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a201e692d0641b69c6215281d41f0c8" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37713 } } }
I20260812 03:03:52.371410 18604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:52.371570 18612 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a201e692d0641b69c6215281d41f0c8. Latest consensus state: current_term: 1 leader_uuid: "4a201e692d0641b69c6215281d41f0c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a201e692d0641b69c6215281d41f0c8" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37713 } } }
I20260812 03:03:52.371740 18612 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:52.372582 18625 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:03:52.375025 18625 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:03:52.380167 18625 catalog_manager.cc:1347] Generated new cluster ID: 2fa404107c3547efaa851db7e51591b7
I20260812 03:03:52.380239 18625 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:03:52.406160 18625 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:03:52.407097 18625 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:03:52.419785 18625 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8: Generated new TSK 0
I20260812 03:03:52.420373 18625 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.96.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
I20260812 03:03:52.456264 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:0
--local_ip_for_outbound_sockets=127.5.96.129
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:37713
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:34069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:52.564121 18644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:52.564347 18644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:52.564388 18644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:52.568234 18644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:52.568282 18644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:52.568338 18644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:03:52.573429 18644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:34069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:37713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:52.574345 18644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:52.575291 18644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:52.584584 18657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:52.584578 18652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:52.584573 18651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:52.584863 18644 server_base.cc:1034] running on GCE node
I20260812 03:03:52.585292 18644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:52.585857 18644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:52.587028 18644 hybrid_clock.cc:648] HybridClock initialized: now 1786503832587006 us; error 32 us; skew 500 ppm
Aug 12 03:03:52 dist-test-slave-6bbx krb5kdc[18455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503832, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:52.589624 18644 init.cc:377] Logged in from keytab as kudu/127.5.96.129@KRBTEST.COM (short username kudu)
I20260812 03:03:52.590437 18644 webserver.cc:460] Webserver started at http://127.5.96.129:33089/ using document root <none> and password file <none>
I20260812 03:03:52.590935 18644 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:52.591001 18644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:52.591197 18644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:52.593137 18644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/data/instance:
uuid: "80086a0ce66045c0a4ffd8d6d2ca23f1"
format_stamp: "Formatted at 2026-08-12 03:03:52 on dist-test-slave-6bbx"
server_key: "2989c1352b3b1bec5b727f6111bb7dd9"
server_key_iv: "1d739553268a2679afbe852122606a3d"
server_key_version: "encryptionkey@0"
I20260812 03:03:52.593644 18644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance:
uuid: "80086a0ce66045c0a4ffd8d6d2ca23f1"
format_stamp: "Formatted at 2026-08-12 03:03:52 on dist-test-slave-6bbx"
server_key: "2989c1352b3b1bec5b727f6111bb7dd9"
server_key_iv: "1d739553268a2679afbe852122606a3d"
server_key_version: "encryptionkey@0"
I20260812 03:03:52.597038 18644 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:03:52.599156 18666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:52.600286 18644 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:03:52.600399 18644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "80086a0ce66045c0a4ffd8d6d2ca23f1"
format_stamp: "Formatted at 2026-08-12 03:03:52 on dist-test-slave-6bbx"
server_key: "2989c1352b3b1bec5b727f6111bb7dd9"
server_key_iv: "1d739553268a2679afbe852122606a3d"
server_key_version: "encryptionkey@0"
I20260812 03:03:52.600485 18644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:52.612066 18644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:52.614825 18644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:52.614981 18644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:52.615489 18644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:52.616271 18644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:52.616317 18644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:52.616365 18644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:52.616393 18644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:52.624373 18644 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:43509
I20260812 03:03:52.624421 18879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:43509 every 8 connection(s)
I20260812 03:03:52.625180 18644 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:03:52.631619 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 18644
I20260812 03:03:52.631750 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance
I20260812 03:03:52.632066 5506 external_mini_cluster.cc:1428] Setting key 03a3eb1f011131c67158554b3b9157f3
Aug 12 03:03:52 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503832, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:52.637400 18597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:52.626762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:59027 (local address 127.5.96.190:37713)
0812 03:03:52.626887 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:52.626891 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:52.627614 (+ 723us) server_negotiation.cc:407] Connection header received
0812 03:03:52.628810 (+ 1196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:52.628813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:52.628861 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:52.628930 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:52.629924 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:52.630433 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:52.631444 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:52.631656 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:52.631804 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:52.634245 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:52.634248 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:52.634250 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:52.634272 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:52.635499 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:52.636007 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:52.636010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:52.636011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:52.636087 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:52.636392 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:52.636395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:52.636397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:52.636568 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:52.636654 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:52.637111 (+ 457us) server_negotiation.cc:299] Negotiation successful
0812 03:03:52.637245 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:03:52.638062 18883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:52.627063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.129:59027)
0812 03:03:52.627474 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:52.627513 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:52.628629 (+ 1116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:52.629066 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:52.629073 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:52.629374 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:52.629814 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:52.629827 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:52.630517 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:52.630519 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:52.631351 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:52.631360 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:52.631839 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:52.631843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:52.632027 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:52.632512 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:52.632527 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:52.634088 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:52.635609 (+ 1521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:52.635614 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:52.635628 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:52.635912 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:52.636168 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:52.636171 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:52.636174 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:52.636304 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:52.636669 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:52.636675 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:52.636894 (+ 219us) client_negotiation.cc:769] Sending connection context
0812 03:03:52.637047 (+ 153us) client_negotiation.cc:240] Negotiation successful
0812 03:03:52.637332 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":122,"threads_started":1}
I20260812 03:03:52.639253 18880 heartbeater.cc:347] Connected to a master server at 127.5.96.190:37713
I20260812 03:03:52.639460 18880 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:52.639921 18880 heartbeater.cc:511] Master 127.5.96.190:37713 requested a full tablet report, sending...
I20260812 03:03:52.641319 18533 ts_manager.cc:194] Registered new tserver with Master: 80086a0ce66045c0a4ffd8d6d2ca23f1 (127.5.96.129:43509)
WARNING: no policy specified for kudu/127.5.96.130@KRBTEST.COM; defaulting to no policy
I20260812 03:03:52.642396 18533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.129@KRBTEST.COM'} at 127.5.96.129:59027
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
I20260812 03:03:52.655652 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:0
--local_ip_for_outbound_sockets=127.5.96.130
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:37713
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:34069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:52.764245 18886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:52.764467 18886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:52.764508 18886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:52.768409 18886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:52.768468 18886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:52.768541 18886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:03:52.773837 18886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:34069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:37713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:52.774724 18886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:52.775619 18886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:52.784313 18902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:52.784305 18893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:52.784304 18894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:52.784523 18886 server_base.cc:1034] running on GCE node
I20260812 03:03:52.784910 18886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:52.785495 18886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:52.786662 18886 hybrid_clock.cc:648] HybridClock initialized: now 1786503832786645 us; error 26 us; skew 500 ppm
Aug 12 03:03:52 dist-test-slave-6bbx krb5kdc[18455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503832, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:52.789238 18886 init.cc:377] Logged in from keytab as kudu/127.5.96.130@KRBTEST.COM (short username kudu)
I20260812 03:03:52.789996 18886 webserver.cc:460] Webserver started at http://127.5.96.130:42871/ using document root <none> and password file <none>
I20260812 03:03:52.790505 18886 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:52.790585 18886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:52.790764 18886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:52.792654 18886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/data/instance:
uuid: "6b5695e105b84dbba469d034e5f29aeb"
format_stamp: "Formatted at 2026-08-12 03:03:52 on dist-test-slave-6bbx"
server_key: "542ebdc84768fc1f2604458fe8c8dfbd"
server_key_iv: "5f1e52f99fd55dfe405a516f024866c8"
server_key_version: "encryptionkey@0"
I20260812 03:03:52.793151 18886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance:
uuid: "6b5695e105b84dbba469d034e5f29aeb"
format_stamp: "Formatted at 2026-08-12 03:03:52 on dist-test-slave-6bbx"
server_key: "542ebdc84768fc1f2604458fe8c8dfbd"
server_key_iv: "5f1e52f99fd55dfe405a516f024866c8"
server_key_version: "encryptionkey@0"
I20260812 03:03:52.796482 18886 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:03:52.798616 18915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:52.799736 18886 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:03:52.799837 18886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "6b5695e105b84dbba469d034e5f29aeb"
format_stamp: "Formatted at 2026-08-12 03:03:52 on dist-test-slave-6bbx"
server_key: "542ebdc84768fc1f2604458fe8c8dfbd"
server_key_iv: "5f1e52f99fd55dfe405a516f024866c8"
server_key_version: "encryptionkey@0"
I20260812 03:03:52.799925 18886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:52.829257 18886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:52.831835 18886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:52.831993 18886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:52.832532 18886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:52.833308 18886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:52.833355 18886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:52.833406 18886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:52.833431 18886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:52.841501 18886 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:45845
I20260812 03:03:52.841526 19126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:45845 every 8 connection(s)
I20260812 03:03:52.842327 18886 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
Aug 12 03:03:52 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503832, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:52.851449 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 18886
I20260812 03:03:52.851554 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance
I20260812 03:03:52.851871 5506 external_mini_cluster.cc:1428] Setting key 7e0497e26d42d6350c2e6fa5c2e2f597
I20260812 03:03:52.855394 18597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:52.844025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:54923 (local address 127.5.96.190:37713)
0812 03:03:52.844186 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:52.844190 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:52.844911 (+ 721us) server_negotiation.cc:407] Connection header received
0812 03:03:52.846228 (+ 1317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:52.846231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:52.846290 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:52.846361 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:52.847459 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:52.848079 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:52.849185 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:52.849402 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:52.849491 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:52.851753 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:52.851757 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:52.851760 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:52.851784 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:52.853533 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:52.853998 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:52.854001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:52.854003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:52.854082 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:52.854396 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:52.854399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:52.854400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:52.854571 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:52.854656 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:52.855120 (+ 464us) server_negotiation.cc:299] Negotiation successful
0812 03:03:52.855238 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 03:03:52.856168 19132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:52.844346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.130:54923)
0812 03:03:52.844775 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:52.844815 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:52.846020 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:52.846533 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:52.846540 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:52.846866 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:52.847318 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:52.847332 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:52.848212 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:52.848216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:52.849065 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:52.849075 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:52.849538 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:52.849542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:52.849709 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:52.850174 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:52.850189 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:52.851570 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:52.853639 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:52.853645 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:52.853660 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:52.853904 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:52.854162 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:52.854166 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:52.854167 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:52.854309 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:52.854671 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:52.854677 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:52.854899 (+ 222us) client_negotiation.cc:769] Sending connection context
0812 03:03:52.855056 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 03:03:52.855364 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":115,"threads_started":1}
I20260812 03:03:52.857168 19127 heartbeater.cc:347] Connected to a master server at 127.5.96.190:37713
I20260812 03:03:52.857409 19127 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:52.857944 19127 heartbeater.cc:511] Master 127.5.96.190:37713 requested a full tablet report, sending...
I20260812 03:03:52.859031 18533 ts_manager.cc:194] Registered new tserver with Master: 6b5695e105b84dbba469d034e5f29aeb (127.5.96.130:45845)
I20260812 03:03:52.859805 18533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.130@KRBTEST.COM'} at 127.5.96.130:54923
WARNING: no policy specified for kudu/127.5.96.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
I20260812 03:03:52.876686 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:0
--local_ip_for_outbound_sockets=127.5.96.131
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:37713
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:34069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:52.984772 19138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:52.984979 19138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:52.985019 19138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:52.988883 19138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:52.988932 19138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:52.988997 19138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:03:52.993968 19138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:34069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:37713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:52.994809 19138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:52.995710 19138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:53.004437 19147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:53.004447 19153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:53.004501 19148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:53.004616 19138 server_base.cc:1034] running on GCE node
I20260812 03:03:53.005080 19138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:53.005661 19138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:53.006825 19138 hybrid_clock.cc:648] HybridClock initialized: now 1786503833006809 us; error 25 us; skew 500 ppm
Aug 12 03:03:53 dist-test-slave-6bbx krb5kdc[18455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503833, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:53.009378 19138 init.cc:377] Logged in from keytab as kudu/127.5.96.131@KRBTEST.COM (short username kudu)
I20260812 03:03:53.010177 19138 webserver.cc:460] Webserver started at http://127.5.96.131:46483/ using document root <none> and password file <none>
I20260812 03:03:53.010664 19138 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:53.010728 19138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:53.010906 19138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:03:53.012802 19138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/data/instance:
uuid: "af78caaa4c134f5f827f2c2fdc70a39e"
format_stamp: "Formatted at 2026-08-12 03:03:53 on dist-test-slave-6bbx"
server_key: "6253d486d738940ab13a53c623d03654"
server_key_iv: "1f0de0fadee5821aba57d07eb07649ff"
server_key_version: "encryptionkey@0"
I20260812 03:03:53.013327 19138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance:
uuid: "af78caaa4c134f5f827f2c2fdc70a39e"
format_stamp: "Formatted at 2026-08-12 03:03:53 on dist-test-slave-6bbx"
server_key: "6253d486d738940ab13a53c623d03654"
server_key_iv: "1f0de0fadee5821aba57d07eb07649ff"
server_key_version: "encryptionkey@0"
I20260812 03:03:53.016652 19138 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260812 03:03:53.018778 19160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:53.019865 19138 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:03:53.019966 19138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "af78caaa4c134f5f827f2c2fdc70a39e"
format_stamp: "Formatted at 2026-08-12 03:03:53 on dist-test-slave-6bbx"
server_key: "6253d486d738940ab13a53c623d03654"
server_key_iv: "1f0de0fadee5821aba57d07eb07649ff"
server_key_version: "encryptionkey@0"
I20260812 03:03:53.020049 19138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:53.041455 19138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:53.043901 19138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:53.044055 19138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:53.044557 19138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:53.045284 19138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:53.045332 19138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:53.045380 19138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:53.045409 19138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:53.053627 19138 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:43585
I20260812 03:03:53.053656 19342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:43585 every 8 connection(s)
I20260812 03:03:53.054404 19138 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:03:53.062198 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 19138
I20260812 03:03:53.062325 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance
I20260812 03:03:53.062594 5506 external_mini_cluster.cc:1428] Setting key 4879feacfd12be209b1079ec09fa1c7e
Aug 12 03:03:53 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503833, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:03:53.066665 18597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:53.055963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:35593 (local address 127.5.96.190:37713)
0812 03:03:53.056086 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:53.056089 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:53.056852 (+ 763us) server_negotiation.cc:407] Connection header received
0812 03:03:53.058082 (+ 1230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:53.058086 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:53.058150 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:53.058218 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:53.059285 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.059846 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.060922 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.061143 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.061233 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:53.063715 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:53.063719 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:53.063722 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:53.063750 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:53.065067 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:53.065503 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:53.065506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:53.065507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:53.065577 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:53.065849 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:53.065850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:53.065851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:53.066006 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:53.066075 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:53.066446 (+ 371us) server_negotiation.cc:299] Negotiation successful
0812 03:03:53.066532 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:03:53.067342 19346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:53.056301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.131:35593)
0812 03:03:53.056720 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:53.056759 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:53.057886 (+ 1127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:53.058389 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:53.058395 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:53.058719 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:53.059147 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:53.059159 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.059966 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.059969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:53.060806 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:53.060815 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.061277 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.061282 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:53.061433 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:53.061858 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:53.061873 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:53.063545 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:03:53.065175 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:53.065179 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:53.065189 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:53.065423 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:53.065651 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:03:53.065653 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:03:53.065654 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:03:53.065784 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:03:53.066073 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:03:53.066077 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:03:53.066274 (+ 197us) client_negotiation.cc:769] Sending connection context
0812 03:03:53.066414 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 03:03:53.066627 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":112,"threads_started":1}
I20260812 03:03:53.068313 19343 heartbeater.cc:347] Connected to a master server at 127.5.96.190:37713
I20260812 03:03:53.068512 19343 heartbeater.cc:464] Registering TS with master...
I20260812 03:03:53.068938 19343 heartbeater.cc:511] Master 127.5.96.190:37713 requested a full tablet report, sending...
I20260812 03:03:53.069844 18533 ts_manager.cc:194] Registered new tserver with Master: af78caaa4c134f5f827f2c2fdc70a39e (127.5.96.131:43585)
I20260812 03:03:53.070399 18533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.131@KRBTEST.COM'} at 127.5.96.131:35593
I20260812 03:03:53.076220 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.5.96.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.96.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.96.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.129.keytab.
Entry for principal oryx/127.5.96.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.129.keytab.
WARNING: no policy specified for oryx/127.5.96.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.96.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.96.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.130.keytab.
Entry for principal oryx/127.5.96.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.130.keytab.
WARNING: no policy specified for oryx/127.5.96.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.96.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.96.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.131.keytab.
Entry for principal oryx/127.5.96.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.131.keytab.
I20260812 03:03:53.139788 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 18644
I20260812 03:03:53.155970 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 18886
I20260812 03:03:53.169369 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 19138
I20260812 03:03:53.183796 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 18476
I20260812 03:03:53.195156 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:37713
--webserver_interface=127.5.96.190
--webserver_port=33737
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:34069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:37713
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:03:53.302791 19355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:53.303028 19355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:53.303072 19355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:53.306958 19355 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:03:53.307008 19355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:53.307025 19355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:53.307041 19355 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:03:53.307070 19355 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:03:53.312163 19355 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:34069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:37713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:37713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=33737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:53.313098 19355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:53.314078 19355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:53.322235 19365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:53.322312 19363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:53.322250 19369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:53.322458 19355 server_base.cc:1034] running on GCE node
I20260812 03:03:53.323033 19355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:53.324079 19355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:53.325237 19355 hybrid_clock.cc:648] HybridClock initialized: now 1786503833325212 us; error 36 us; skew 500 ppm
Aug 12 03:03:53 dist-test-slave-6bbx krb5kdc[18455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503833, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:53.327876 19355 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:03:53.328644 19355 webserver.cc:460] Webserver started at http://127.5.96.190:33737/ using document root <none> and password file <none>
I20260812 03:03:53.329252 19355 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:53.329339 19355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:53.333069 19355 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 03:03:53.335135 19383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:53.336333 19355 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:53.336433 19355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "4a201e692d0641b69c6215281d41f0c8"
format_stamp: "Formatted at 2026-08-12 03:03:52 on dist-test-slave-6bbx"
server_key: "ffcaa4219a2c163e5ad3e8eb846b7c70"
server_key_iv: "f076dad2641ec5b06fbda9628f25b99b"
server_key_version: "encryptionkey@0"
I20260812 03:03:53.336908 19355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:53.360401 19355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:53.363288 19355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:53.363430 19355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:53.370002 19355 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:37713
I20260812 03:03:53.370018 19478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:37713 every 8 connection(s)
I20260812 03:03:53.371058 19355 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:03:53.375118 19480 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:03:53.376758 19480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8: Bootstrap starting.
I20260812 03:03:53.381201 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 19355
I20260812 03:03:53.381613 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:43509
--local_ip_for_outbound_sockets=127.5.96.129
--tserver_master_addrs=127.5.96.190:37713
--webserver_port=33089
--webserver_interface=127.5.96.129
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:34069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.96.129
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.129.keytab with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:03:53.396654 19480 log.cc:824] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8: Log is configured to *not* fsync() on all Append() calls
I20260812 03:03:53.404980 19480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:03:53.405391 19480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8: Bootstrap complete.
I20260812 03:03:53.410945 19480 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a201e692d0641b69c6215281d41f0c8" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37713 } }
I20260812 03:03:53.411458 19480 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a201e692d0641b69c6215281d41f0c8, State: Initialized, Role: FOLLOWER
I20260812 03:03:53.412040 19480 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a201e692d0641b69c6215281d41f0c8" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37713 } }
I20260812 03:03:53.412189 19480 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:03:53.412256 19480 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:03:53.412359 19480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:03:53.414788 19480 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a201e692d0641b69c6215281d41f0c8" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37713 } }
I20260812 03:03:53.415148 19480 leader_election.cc:302] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a201e692d0641b69c6215281d41f0c8; no voters:
I20260812 03:03:53.415486 19480 leader_election.cc:288] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 03:03:53.415597 19486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:03:53.415907 19486 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [term 2 LEADER]: Becoming Leader. State: Replica: 4a201e692d0641b69c6215281d41f0c8, State: Running, Role: LEADER
I20260812 03:03:53.416244 19486 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a201e692d0641b69c6215281d41f0c8" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37713 } }
I20260812 03:03:53.416752 19480 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:03:53.417568 19497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a201e692d0641b69c6215281d41f0c8. Latest consensus state: current_term: 2 leader_uuid: "4a201e692d0641b69c6215281d41f0c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a201e692d0641b69c6215281d41f0c8" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37713 } } }
I20260812 03:03:53.417711 19497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:53.418304 19489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4a201e692d0641b69c6215281d41f0c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a201e692d0641b69c6215281d41f0c8" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37713 } } }
I20260812 03:03:53.418388 19489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8 [sys.catalog]: This master's current role is: LEADER
I20260812 03:03:53.418763 19510 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:03:53.422256 19510 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:03:53.422901 19510 catalog_manager.cc:1259] Loaded cluster ID: 2fa404107c3547efaa851db7e51591b7
I20260812 03:03:53.422964 19510 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:03:53.425383 19510 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:03:53.426049 19510 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4a201e692d0641b69c6215281d41f0c8: Loaded TSK: 0
I20260812 03:03:53.427260 19510 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:03:53.506014 19484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:53.506253 19484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:53.506291 19484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:53.510159 19484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:53.510210 19484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:53.510265 19484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:03:53.515223 19484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:34069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.129.keytab
--principal=oryx/127.5.96.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:43509
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=33089
--tserver_master_addrs=127.5.96.190:37713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:53.516181 19484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:53.517145 19484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:53.526556 19539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:53.526690 19535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:53.526708 19542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:53.527146 19484 server_base.cc:1034] running on GCE node
I20260812 03:03:53.527587 19484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:53.528360 19484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:53.529552 19484 hybrid_clock.cc:648] HybridClock initialized: now 1786503833529527 us; error 35 us; skew 500 ppm
Aug 12 03:03:53 dist-test-slave-6bbx krb5kdc[18455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503833, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:53.535706 19484 init.cc:377] Logged in from keytab as oryx/127.5.96.129@KRBTEST.COM (short username oryx)
I20260812 03:03:53.536779 19484 webserver.cc:460] Webserver started at http://127.5.96.129:33089/ using document root <none> and password file <none>
I20260812 03:03:53.537426 19484 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:53.537523 19484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:53.542557 19484 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 03:03:53.545107 19557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:53.546430 19484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:03:53.546541 19484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "80086a0ce66045c0a4ffd8d6d2ca23f1"
format_stamp: "Formatted at 2026-08-12 03:03:52 on dist-test-slave-6bbx"
server_key: "2989c1352b3b1bec5b727f6111bb7dd9"
server_key_iv: "1d739553268a2679afbe852122606a3d"
server_key_version: "encryptionkey@0"
I20260812 03:03:53.547183 19484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:53.565680 19484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:53.568289 19484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:53.568429 19484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:53.568926 19484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:53.569674 19484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:53.569722 19484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:53.569758 19484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:53.569779 19484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:53.577845 19484 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:43509
I20260812 03:03:53.577870 19781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:43509 every 8 connection(s)
I20260812 03:03:53.578763 19484 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:03:53.579468 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 19484
I20260812 03:03:53.579960 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:45845
--local_ip_for_outbound_sockets=127.5.96.130
--tserver_master_addrs=127.5.96.190:37713
--webserver_port=42871
--webserver_interface=127.5.96.130
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:34069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.96.130
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.130.keytab with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:03:53 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:53 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:03:53.591482 19791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:53.580917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.129:39487)
0812 03:03:53.581862 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:53.581902 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:53.582693 (+ 791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:53.583815 (+ 1122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:53.583823 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:53.584210 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:53.584653 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:53.584668 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.585995 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.585998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:53.586854 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:53.586864 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.587601 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.587606 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:53.587809 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:53.588350 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:53.588365 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:53.590564 (+ 2199us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":760,"thread_start_us":622,"threads_started":1}
W20260812 03:03:53.591482 19786 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:53.580852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:39487 (local address 127.5.96.190:37713)
0812 03:03:53.581357 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:53.581379 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:53.582016 (+ 637us) server_negotiation.cc:407] Connection header received
0812 03:03:53.583073 (+ 1057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:53.583083 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:53.583378 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:53.583699 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:53.584823 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.585843 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.586977 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.587436 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.587586 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:53.590802 (+ 3216us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.129:39487: BlockingRecv error: recv got EOF from 127.5.96.129:39487 (error 108)
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":122,"threads_started":1}
W20260812 03:03:53.592197 19783 heartbeater.cc:644] Failed to heartbeat to 127.5.96.190:37713 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.96.190:37713: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Aug 12 03:03:53 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:53 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:03:53.597697 19791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:53.592607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.129:55761)
0812 03:03:53.592758 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:53.592772 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:53.592837 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:53.593037 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:53.593039 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:53.593206 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:53.593311 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:53.593317 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.593974 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.593977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:53.594856 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:53.594866 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.595325 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.595328 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:53.595420 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:53.595857 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:53.595871 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:53.597573 (+ 1702us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93}
W20260812 03:03:53.597806 19786 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:53.592685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:55761 (local address 127.5.96.190:37713)
0812 03:03:53.592784 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:53.592787 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:53.592802 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:03:53.592909 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:53.592910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:53.592948 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:53.593010 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:53.593418 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.593888 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.594971 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.595223 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.595315 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:53.597708 (+ 2393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.129:55761: BlockingRecv error: recv got EOF from 127.5.96.129:55761 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 12 03:03:53 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:53 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:03:53.602820 19791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:53.598257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.129:40393)
0812 03:03:53.598392 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:53.598404 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:53.598472 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:53.598696 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:53.598699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:53.598827 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:53.598893 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:53.598898 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.599561 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.599564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:53.600118 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:53.600127 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.600517 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.600520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:53.600596 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:53.600972 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:53.600983 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:53.602714 (+ 1731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":85}
W20260812 03:03:53.602927 19786 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:53.598352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:40393 (local address 127.5.96.190:37713)
0812 03:03:53.598467 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:53.598470 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:53.598483 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:03:53.598550 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:53.598552 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:53.598605 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:53.598666 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:53.598994 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.599455 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.600224 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.600424 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.600506 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:53.602831 (+ 2325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.129:40393: BlockingRecv error: recv got EOF from 127.5.96.129:40393 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260812 03:03:53.603018 19783 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:03:53.702565 19787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:53.702832 19787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:53.702890 19787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:53.706898 19787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:53.706946 19787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:53.707017 19787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:03:53.712054 19787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:34069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.130.keytab
--principal=oryx/127.5.96.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:45845
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=42871
--tserver_master_addrs=127.5.96.190:37713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:53.712976 19787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:53.713905 19787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:53.723070 19807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:53.723044 19798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:53.723423 19787 server_base.cc:1034] running on GCE node
W20260812 03:03:53.723044 19800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:53.724066 19787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:53.724627 19787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:53.725780 19787 hybrid_clock.cc:648] HybridClock initialized: now 1786503833725764 us; error 25 us; skew 500 ppm
Aug 12 03:03:53 dist-test-slave-6bbx krb5kdc[18455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503833, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:53.728323 19787 init.cc:377] Logged in from keytab as oryx/127.5.96.130@KRBTEST.COM (short username oryx)
I20260812 03:03:53.729097 19787 webserver.cc:460] Webserver started at http://127.5.96.130:42871/ using document root <none> and password file <none>
I20260812 03:03:53.729609 19787 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:53.729688 19787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:53.733300 19787 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.002s
I20260812 03:03:53.735162 19821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:53.736286 19787 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:53.736389 19787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "6b5695e105b84dbba469d034e5f29aeb"
format_stamp: "Formatted at 2026-08-12 03:03:52 on dist-test-slave-6bbx"
server_key: "542ebdc84768fc1f2604458fe8c8dfbd"
server_key_iv: "5f1e52f99fd55dfe405a516f024866c8"
server_key_version: "encryptionkey@0"
I20260812 03:03:53.736877 19787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:53.754268 19787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:53.756805 19787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:53.756945 19787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:53.757452 19787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:53.758181 19787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:53.758237 19787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:53.758273 19787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:53.758293 19787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:53.766206 19787 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:45845
I20260812 03:03:53.766228 20026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:45845 every 8 connection(s)
I20260812 03:03:53.767096 19787 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:03:53.767400 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 19787
I20260812 03:03:53.767918 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:43585
--local_ip_for_outbound_sockets=127.5.96.131
--tserver_master_addrs=127.5.96.190:37713
--webserver_port=46483
--webserver_interface=127.5.96.131
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:34069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.96.131
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.131.keytab with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:03:53 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:53 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 03:03:53.777981 19786 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:53.768806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:47295 (local address 127.5.96.190:37713)
0812 03:03:53.768922 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:53.768925 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:53.769895 (+ 970us) server_negotiation.cc:407] Connection header received
0812 03:03:53.771486 (+ 1591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:53.771489 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:53.771540 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:53.771601 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:53.772714 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.773240 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.774311 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.774533 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.774620 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:53.777876 (+ 3256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.130:47295: BlockingRecv error: recv got EOF from 127.5.96.130:47295 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:03:53.778704 20029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:53.769258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.130:47295)
0812 03:03:53.769729 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:53.769790 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:53.771248 (+ 1458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:53.771775 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:53.771782 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:53.772110 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:53.772559 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:53.772571 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.773359 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.773362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:53.774183 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:53.774192 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.774668 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.774672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:53.774826 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:53.775307 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:53.775323 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:53.777761 (+ 2438us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":132,"threads_started":1}
W20260812 03:03:53.779136 20027 heartbeater.cc:644] Failed to heartbeat to 127.5.96.190:37713 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.96.190:37713: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Aug 12 03:03:53 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:53 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:03:53.783929 20029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:53.779515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.130:47213)
0812 03:03:53.779649 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:53.779660 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:53.779749 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:53.779964 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:53.779967 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:53.780101 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:53.780179 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:53.780184 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.780859 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.780862 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:53.781381 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:53.781388 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.781802 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.781805 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:53.781887 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:53.782277 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:53.782288 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:53.783818 (+ 1530us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":84}
W20260812 03:03:53.784016 19786 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:53.779630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:47213 (local address 127.5.96.190:37713)
0812 03:03:53.779791 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:53.779796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:53.779808 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:03:53.779841 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:53.779844 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:53.779877 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:53.779938 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:53.780289 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.780764 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.781484 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.781690 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.781791 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:53.783921 (+ 2130us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.130:47213: BlockingRecv error: recv got EOF from 127.5.96.130:47213 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 12 03:03:53 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:53 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:03:53.789080 20029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:53.784502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.130:51335)
0812 03:03:53.784593 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:53.784606 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:53.784669 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:53.784876 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:53.784878 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:53.784988 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:53.785048 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:53.785052 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.785997 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.785999 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:53.786473 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:53.786479 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.786966 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.786969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:53.787018 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:53.787390 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:53.787400 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:53.788980 (+ 1580us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260812 03:03:53.789199 19786 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:53.784571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:51335 (local address 127.5.96.190:37713)
0812 03:03:53.784672 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:53.784676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:53.784689 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:03:53.784745 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:53.784748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:53.784789 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:53.784872 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:53.785142 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.785911 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.786557 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.786886 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.786973 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:53.789104 (+ 2131us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.130:51335: BlockingRecv error: recv got EOF from 127.5.96.130:51335 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260812 03:03:53.789450 20027 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:03:53.880254 20030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:03:53.880473 20030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:03:53.880528 20030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:03:53.884402 20030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:03:53.884450 20030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:03:53.884526 20030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:03:53.889439 20030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:34069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/krb5kdc/oryx_127.5.96.131.keytab
--principal=oryx/127.5.96.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:43585
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=46483
--tserver_master_addrs=127.5.96.190:37713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:03:53.890343 20030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:03:53.891264 20030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:03:53.900285 20043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:53.900277 20046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:03:53.900277 20039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:03:53.900353 20030 server_base.cc:1034] running on GCE node
I20260812 03:03:53.900919 20030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:03:53.901463 20030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:03:53.902607 20030 hybrid_clock.cc:648] HybridClock initialized: now 1786503833902585 us; error 32 us; skew 500 ppm
Aug 12 03:03:53 dist-test-slave-6bbx krb5kdc[18455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503833, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.96.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:03:53.905093 20030 init.cc:377] Logged in from keytab as oryx/127.5.96.131@KRBTEST.COM (short username oryx)
I20260812 03:03:53.905876 20030 webserver.cc:460] Webserver started at http://127.5.96.131:46483/ using document root <none> and password file <none>
I20260812 03:03:53.906373 20030 fs_manager.cc:359] Metadata directory not provided
I20260812 03:03:53.906448 20030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:03:53.910243 20030 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:03:53.912159 20056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:53.913307 20030 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:03:53.913425 20030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "af78caaa4c134f5f827f2c2fdc70a39e"
format_stamp: "Formatted at 2026-08-12 03:03:53 on dist-test-slave-6bbx"
server_key: "6253d486d738940ab13a53c623d03654"
server_key_iv: "1f0de0fadee5821aba57d07eb07649ff"
server_key_version: "encryptionkey@0"
I20260812 03:03:53.913908 20030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:03:53.972707 20030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:03:53.975286 20030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:03:53.975450 20030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:03:53.976012 20030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:03:53.976789 20030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:03:53.976838 20030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:53.976891 20030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:03:53.976920 20030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:03:53.985155 20030 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:43585
I20260812 03:03:53.985184 20267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:43585 every 8 connection(s)
I20260812 03:03:53.986065 20030 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:03:53 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:53 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 03:03:53.995718 19786 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:53.987760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:49677 (local address 127.5.96.190:37713)
0812 03:03:53.987889 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:53.987893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:53.988660 (+ 767us) server_negotiation.cc:407] Connection header received
0812 03:03:53.989875 (+ 1215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:53.989878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:53.989926 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:53.989994 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:53.991010 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.991505 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.992559 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.992779 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.992848 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:53.995576 (+ 2728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.131:49677: BlockingRecv error: recv got EOF from 127.5.96.131:49677 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:03:53.995958 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 20030
I20260812 03:03:53.996336 20272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:53.988099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.131:49677)
0812 03:03:53.988512 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:53.988553 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:53.989685 (+ 1132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:53.990121 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:53.990128 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:53.990458 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:53.990899 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:53.990911 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.991591 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.991593 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:53.992459 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:53.992468 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.992880 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.992882 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:53.993006 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:53.993457 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:53.993471 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:53.995458 (+ 1987us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":120,"threads_started":1}
W20260812 03:03:53.997097 20269 heartbeater.cc:644] Failed to heartbeat to 127.5.96.190:37713 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.96.190:37713: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Aug 12 03:03:53 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:53 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:03:54.001843 20272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:53.997446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.131:46491)
0812 03:03:53.997579 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:53.997592 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:53.997665 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:53.997951 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:53.997954 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:53.998103 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:53.998180 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:53.998185 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.998866 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.998870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:53.999383 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:53.999390 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.999839 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.999842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:53.999927 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:54.000239 (+ 312us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:54.000259 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:54.001738 (+ 1479us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260812 03:03:54.001982 20275 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:53.997551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:46491 (local address 127.5.96.190:37713)
0812 03:03:53.997768 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:53.997772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:53.997782 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:03:53.997811 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:53.997813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:53.997843 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:53.997918 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:53.998284 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.998773 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.999485 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.999728 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.999815 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:54.001869 (+ 2054us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.131:46491: BlockingRecv error: recv got EOF from 127.5.96.131:46491 (error 108)
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":157,"thread_start_us":82,"threads_started":1}
I20260812 03:03:54.004678 19786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:53.996881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56680 (local address 127.5.96.190:37713)
0812 03:03:53.996998 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:53.997002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:53.997015 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:03:53.997051 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:53.997054 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:53.997092 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:53.997154 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:53.997937 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.998466 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.999269 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:53.999475 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:53.999556 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:54.000861 (+ 1305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:03:54.000866 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:03:54.000880 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:03:54.000906 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:03:54.002723 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:54.003114 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:54.003120 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:54.003126 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:54.003241 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:03:54.003511 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:03:54.003514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:03:54.003516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:03:54.003882 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:03:54.004072 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 03:03:54.004321 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 03:03:54.004541 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":45}
Aug 12 03:03:54 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:54 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:03:54.006702 20272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:54.002346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.131:35129)
0812 03:03:54.002431 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:54.002443 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:54.002511 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:54.002722 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:54.002725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:54.002857 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:54.002918 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:54.002923 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:54.003561 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:54.003564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:54.004091 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:54.004100 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:54.004515 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:54.004519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:54.004595 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:54.004981 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:54.004994 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:54.006601 (+ 1607us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260812 03:03:54.006882 20275 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:54.002426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:35129 (local address 127.5.96.190:37713)
0812 03:03:54.002560 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:54.002564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:54.002575 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:03:54.002603 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:54.002606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:54.002639 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:54.002697 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:54.003015 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:54.003468 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:54.004195 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:54.004413 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:54.004509 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:54.006694 (+ 2185us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.131:35129: BlockingRecv error: recv got EOF from 127.5.96.131:35129 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260812 03:03:54.007232 20269 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 03:03:54 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:54 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:03:54.608400 20276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:54.603524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.129:41149)
0812 03:03:54.603806 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:54.603819 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:54.603900 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:54.604109 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:54.604111 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:54.604294 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:54.604399 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:54.604405 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:54.605070 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:54.605072 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:54.605630 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:54.605637 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:54.605985 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:54.605987 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:54.606043 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:54.606514 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:54.606526 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:54.608281 (+ 1755us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":78,"threads_started":1}
W20260812 03:03:54.608532 20278 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:54.603617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:41149 (local address 127.5.96.190:37713)
0812 03:03:54.603856 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:54.603859 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:54.603872 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:03:54.603971 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:54.603973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:54.604016 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:54.604086 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:54.604481 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:54.604998 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:54.605708 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:54.605915 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:54.605975 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:54.608415 (+ 2440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.129:41149: BlockingRecv error: recv got EOF from 127.5.96.129:41149 (error 108)
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":77,"threads_started":1}
Aug 12 03:03:54 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:54 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:03:54.794817 20279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:54.789934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.130:41143)
0812 03:03:54.790169 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:54.790182 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:54.790278 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:54.790495 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:54.790497 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:54.790663 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:54.790756 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:54.790761 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:54.791430 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:54.791432 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:54.792045 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:54.792055 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:54.792423 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:54.792426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:54.792482 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:54.792966 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:54.792978 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:54.794669 (+ 1691us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":67,"threads_started":1}
W20260812 03:03:54.794929 20278 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:54.790013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:41143 (local address 127.5.96.190:37713)
0812 03:03:54.790175 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:54.790179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:54.790253 (+ 74us) server_negotiation.cc:407] Connection header received
0812 03:03:54.790358 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:54.790361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:54.790412 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:54.790491 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:54.790838 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:54.791348 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:54.792136 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:54.792345 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:54.792414 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:54.794830 (+ 2416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.130:41143: BlockingRecv error: recv got EOF from 127.5.96.130:41143 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 12 03:03:55 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:55 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:03:55.012898 20283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:55.007793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.131:35449)
0812 03:03:55.008024 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:55.008037 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:55.008127 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:55.008338 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:55.008339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:55.008504 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:55.008598 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:55.008603 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:55.009430 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:55.009434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:55.010222 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:55.010232 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:55.010633 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:55.010635 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:55.010690 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:55.011142 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:55.011154 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:55.012784 (+ 1630us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":69,"threads_started":1}
W20260812 03:03:55.013150 20278 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:55.007836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:35449 (local address 127.5.96.190:37713)
0812 03:03:55.007962 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:55.007965 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:55.008085 (+ 120us) server_negotiation.cc:407] Connection header received
0812 03:03:55.008201 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:55.008203 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:55.008257 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:55.008313 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:55.008676 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:55.009311 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:55.010329 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:55.010548 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:55.010624 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:55.012967 (+ 2343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.131:35449: BlockingRecv error: recv got EOF from 127.5.96.131:35449 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 03:03:55 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:55 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:03:55.614323 20284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:55.609179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.129:58771)
0812 03:03:55.609435 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:55.609448 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:55.609536 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:55.609742 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:55.609744 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:55.609933 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:55.610035 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:55.610040 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:55.610723 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:55.610725 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:55.611278 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:55.611285 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:55.611644 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:55.611647 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:55.611764 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:55.612215 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:55.612226 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:55.614122 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":84,"threads_started":1}
W20260812 03:03:55.614379 20285 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:55.609313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:58771 (local address 127.5.96.190:37713)
0812 03:03:55.609558 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:55.609562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:55.609574 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:03:55.609608 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:55.609610 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:55.609648 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:55.609733 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:55.610119 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:55.610644 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:55.611356 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:55.611569 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:55.611635 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:55.614266 (+ 2631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.129:58771: BlockingRecv error: recv got EOF from 127.5.96.129:58771 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":96,"threads_started":1}
Aug 12 03:03:55 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:55 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:03:55.800452 20287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:55.795536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.130:57817)
0812 03:03:55.795794 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:55.795806 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:55.795900 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:55.796095 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:55.796097 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:55.796282 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:55.796378 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:55.796384 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:55.797067 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:55.797069 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:55.797624 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:55.797631 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:55.797988 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:55.797990 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:55.798046 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:55.798503 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:55.798514 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:55.800309 (+ 1795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":71,"threads_started":1}
W20260812 03:03:55.800591 20285 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:55.795617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:57817 (local address 127.5.96.190:37713)
0812 03:03:55.795764 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:55.795767 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:55.795855 (+ 88us) server_negotiation.cc:407] Connection header received
0812 03:03:55.795974 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:55.795976 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:55.796018 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:55.796073 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:55.796461 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:55.796987 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:55.797700 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:55.797915 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:55.797980 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:55.800467 (+ 2487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.130:57817: BlockingRecv error: recv got EOF from 127.5.96.130:57817 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 03:03:56 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:56 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:03:56.019083 20288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:56.013616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.131:56507)
0812 03:03:56.013908 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:56.013924 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:56.014034 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:56.014285 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:56.014289 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:56.014512 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:56.014633 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:56.014642 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:56.015367 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:56.015370 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:56.015993 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:56.016002 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:56.016493 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:56.016497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:56.016584 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:56.017036 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:56.017049 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:56.018944 (+ 1895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":82,"threads_started":1}
W20260812 03:03:56.019177 20285 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:56.013794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:56507 (local address 127.5.96.190:37713)
0812 03:03:56.013908 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:56.013912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:56.013995 (+ 83us) server_negotiation.cc:407] Connection header received
0812 03:03:56.014119 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:56.014121 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:56.014168 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:56.014236 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:56.014733 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:56.015264 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:56.016101 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:56.016388 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:56.016475 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:56.019075 (+ 2600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.131:56507: BlockingRecv error: recv got EOF from 127.5.96.131:56507 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 12 03:03:56 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:56 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:03:56.620429 20289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:56.614991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.129:49271)
0812 03:03:56.615266 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:56.615279 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:56.615365 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:56.615573 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:56.615575 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:56.615775 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:56.615878 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:56.615884 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:56.616675 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:56.616679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:56.617373 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:56.617385 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:56.617903 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:56.617906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:56.617971 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:56.618436 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:56.618450 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:56.620313 (+ 1863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":91,"threads_started":1}
W20260812 03:03:56.620561 20291 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:56.615064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:49271 (local address 127.5.96.190:37713)
0812 03:03:56.615299 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:56.615302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:56.615318 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:03:56.615436 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:56.615438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:56.615478 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:56.615549 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:56.615960 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:56.616580 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:56.617475 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:56.617804 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:56.617883 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:56.620432 (+ 2549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.129:49271: BlockingRecv error: recv got EOF from 127.5.96.129:49271 (error 108)
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":82,"threads_started":1}
Aug 12 03:03:56 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:56 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:03:56.805943 20292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:56.801091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.130:49421)
0812 03:03:56.801346 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:56.801359 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:56.801455 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:56.801655 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:56.801657 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:56.801827 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:56.801925 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:56.801931 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:56.802621 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:56.802624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:56.803166 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:56.803173 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:56.803530 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:56.803532 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:56.803588 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:56.804075 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:56.804087 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:56.805799 (+ 1712us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":71,"threads_started":1}
W20260812 03:03:56.806160 20291 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:56.801156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:49421 (local address 127.5.96.190:37713)
0812 03:03:56.801293 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:56.801297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:56.801416 (+ 119us) server_negotiation.cc:407] Connection header received
0812 03:03:56.801528 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:56.801530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:56.801574 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:56.801629 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:56.802005 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:56.802542 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:56.803244 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:56.803457 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:56.803522 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:56.805943 (+ 2421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.130:49421: BlockingRecv error: recv got EOF from 127.5.96.130:49421 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 12 03:03:57 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:57 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:03:57.024852 20293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:57.019738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.131:40835)
0812 03:03:57.019991 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:57.020004 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:57.020100 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:57.020304 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:57.020306 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:57.020483 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:57.020579 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:57.020584 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:57.021278 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:57.021280 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:57.021828 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:57.021836 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:57.022325 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:57.022328 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:57.022395 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:57.022905 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:57.022917 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:57.024735 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":85,"threads_started":1}
W20260812 03:03:57.024991 20291 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:57.019779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:40835 (local address 127.5.96.190:37713)
0812 03:03:57.019916 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:57.019920 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:57.020068 (+ 148us) server_negotiation.cc:407] Connection header received
0812 03:03:57.020166 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:57.020169 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:57.020216 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:57.020282 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:57.020658 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:57.021182 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:57.021963 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:57.022221 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:57.022310 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:57.024872 (+ 2562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.131:40835: BlockingRecv error: recv got EOF from 127.5.96.131:40835 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 03:03:57 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:57 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:03:57.630189 20294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:57.621107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.129:45195)
0812 03:03:57.621369 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:57.621382 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:57.621468 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:57.621684 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:57.621686 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:57.621865 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:57.621967 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:57.621973 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:57.622655 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:57.622657 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:57.623206 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:57.623213 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:57.623575 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:57.623577 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:57.623634 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:57.628319 (+ 4685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:57.628331 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:57.630037 (+ 1706us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":86,"threads_started":1}
W20260812 03:03:57.630299 20296 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:57.621184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:45195 (local address 127.5.96.190:37713)
0812 03:03:57.621404 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:57.621407 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:57.621421 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:03:57.621545 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:57.621548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:57.621588 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:57.621660 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:57.622047 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:57.622576 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:57.623287 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:57.623500 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:57.623565 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:57.630182 (+ 6617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.129:45195: BlockingRecv error: recv got EOF from 127.5.96.129:45195 (error 108)
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":59,"threads_started":1}
Aug 12 03:03:57 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:57 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:03:57.811815 20304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:57.806628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.130:52803)
0812 03:03:57.806869 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:57.806883 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:57.806980 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:57.807179 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:57.807182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:57.807361 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:57.807459 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:57.807465 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:57.808183 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:57.808187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:57.808990 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:57.809000 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:57.809472 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:57.809475 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:57.809538 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:57.809976 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:57.809986 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:57.811685 (+ 1699us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":81,"threads_started":1}
W20260812 03:03:57.812016 20296 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:57.806696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:52803 (local address 127.5.96.190:37713)
0812 03:03:57.806833 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:57.806836 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:57.806944 (+ 108us) server_negotiation.cc:407] Connection header received
0812 03:03:57.807049 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:57.807051 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:57.807093 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:57.807156 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:57.807539 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:57.808080 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:57.809106 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:57.809377 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:57.809455 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:57.811893 (+ 2438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.130:52803: BlockingRecv error: recv got EOF from 127.5.96.130:52803 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 03:03:58 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:58 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:03:58.030262 20305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:58.025488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.131:58831)
0812 03:03:58.025742 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:58.025756 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:58.025850 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:58.026045 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:58.026047 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:58.026217 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:58.026325 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:58.026330 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:58.026996 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:58.026998 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:58.027547 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:58.027553 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:58.027947 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:58.027950 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:58.028015 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:58.028472 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:58.028484 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:58.030145 (+ 1661us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":79,"threads_started":1}
W20260812 03:03:58.030369 20296 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:58.025555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:58831 (local address 127.5.96.190:37713)
0812 03:03:58.025684 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:58.025689 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:58.025806 (+ 117us) server_negotiation.cc:407] Connection header received
0812 03:03:58.025921 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:58.025923 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:58.025967 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:58.026021 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:58.026406 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:58.026921 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:58.027623 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:58.027858 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:58.027924 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:58.030270 (+ 2346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.131:58831: BlockingRecv error: recv got EOF from 127.5.96.131:58831 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 12 03:03:58 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:58 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:03:58.636091 20308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:58.630946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.129:52871)
0812 03:03:58.631206 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:58.631228 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:58.631314 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:58.631523 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:58.631525 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:58.631746 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:58.631847 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:58.631852 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:58.632746 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:58.632750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:58.633362 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:58.633372 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:58.633784 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:58.633786 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:58.633848 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:58.634276 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:58.634286 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:58.635975 (+ 1689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":70,"threads_started":1}
W20260812 03:03:58.636231 20309 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:58.631007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:52871 (local address 127.5.96.190:37713)
0812 03:03:58.631264 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:58.631267 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:58.631279 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:03:58.631388 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:58.631390 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:58.631431 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:58.631502 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:58.631954 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:58.632626 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:58.633465 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:58.633696 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:58.633780 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:58.636102 (+ 2322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.129:52871: BlockingRecv error: recv got EOF from 127.5.96.129:52871 (error 108)
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":78,"threads_started":1}
Aug 12 03:03:58 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:58 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:03:58.817392 20310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:58.812541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.130:35495)
0812 03:03:58.812793 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:58.812806 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:58.812898 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:58.813103 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:58.813105 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:58.813298 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:58.813399 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:58.813404 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:58.814069 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:58.814072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:58.814629 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:58.814636 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:58.814988 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:58.814990 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:58.815045 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:58.815498 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:58.815510 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:58.817279 (+ 1769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":84,"threads_started":1}
W20260812 03:03:58.817490 20309 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:58.812676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:35495 (local address 127.5.96.190:37713)
0812 03:03:58.812806 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:58.812809 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:58.812876 (+ 67us) server_negotiation.cc:407] Connection header received
0812 03:03:58.812966 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:58.812969 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:58.813011 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:58.813073 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:58.813483 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:58.813992 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:58.814709 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:58.814914 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:58.814978 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:58.817392 (+ 2414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.130:35495: BlockingRecv error: recv got EOF from 127.5.96.130:35495 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 03:03:59 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:59 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:03:59.036034 20311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:59.030949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.131:40113)
0812 03:03:59.031211 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:59.031225 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:59.031329 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:59.031574 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:59.031578 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:59.031800 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:59.031917 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:59.031927 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:59.032845 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:59.032848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:59.033407 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:59.033417 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:59.033777 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:59.033780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:59.033844 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:59.034273 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:59.034284 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:59.035920 (+ 1636us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":73,"threads_started":1}
W20260812 03:03:59.036139 20309 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:59.031021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:40113 (local address 127.5.96.190:37713)
0812 03:03:59.031171 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:59.031175 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:59.031288 (+ 113us) server_negotiation.cc:407] Connection header received
0812 03:03:59.031410 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:59.031412 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:59.031454 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:59.031517 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:59.032039 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:59.032743 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:59.033494 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:59.033701 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:59.033765 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:59.036042 (+ 2277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.131:40113: BlockingRecv error: recv got EOF from 127.5.96.131:40113 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 12 03:03:59 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:59 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:03:59.641706 20312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:59.636728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.129:60467)
0812 03:03:59.636985 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:59.636998 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:59.637083 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:59.637299 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:59.637301 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:59.637481 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:59.637580 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:59.637585 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:59.638268 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:59.638270 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:59.638821 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:59.638828 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:59.639192 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:59.639194 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:59.639254 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:59.639790 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:59.639802 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:59.641592 (+ 1790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":69,"threads_started":1}
W20260812 03:03:59.641848 20315 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:59.636783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:60467 (local address 127.5.96.190:37713)
0812 03:03:59.637033 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:59.637036 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:59.637049 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:03:59.637160 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:59.637162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:59.637203 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:59.637279 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:59.637658 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:59.638190 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:59.638900 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:59.639117 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:59.639182 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:59.641718 (+ 2536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.129:60467: BlockingRecv error: recv got EOF from 127.5.96.129:60467 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":76,"threads_started":1}
Aug 12 03:03:59 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:03:59 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:03:59.823361 20319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:03:59.818155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.130:45917)
0812 03:03:59.818441 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 03:03:59.818453 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:03:59.818562 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:03:59.818814 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:03:59.818818 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:03:59.819025 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:03:59.819121 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:59.819127 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:59.819917 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:59.819920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:59.820478 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:03:59.820486 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:03:59.820924 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:03:59.820928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:03:59.821012 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:59.821468 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:03:59.821481 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:03:59.823246 (+ 1765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":93,"threads_started":1}
W20260812 03:03:59.823488 20315 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:03:59.818216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:45917 (local address 127.5.96.190:37713)
0812 03:03:59.818372 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 03:03:59.818377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:03:59.818522 (+ 145us) server_negotiation.cc:407] Connection header received
0812 03:03:59.818650 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:03:59.818653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:03:59.818701 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:03:59.818765 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:03:59.819232 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:59.819817 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:59.820588 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:03:59.820823 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:03:59.820910 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:03:59.823371 (+ 2461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.130:45917: BlockingRecv error: recv got EOF from 127.5.96.130:45917 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 12 03:04:00 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:04:00 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:04:00.042057 20320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:00.036772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.131:59125)
0812 03:04:00.037033 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:00.037047 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:00.037158 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:00.037387 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:00.037389 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:00.037574 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:00.037674 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:00.037680 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:00.038399 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:00.038402 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:00.038958 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:00.038964 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:00.039386 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:00.039389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:00.039463 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:00.039968 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:00.039983 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:00.041905 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":88,"threads_started":1}
W20260812 03:04:00.042272 20315 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:00.036820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:59125 (local address 127.5.96.190:37713)
0812 03:04:00.036941 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:00.036946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:00.037150 (+ 204us) server_negotiation.cc:407] Connection header received
0812 03:04:00.037245 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:00.037247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:00.037296 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:00.037360 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:00.037772 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:00.038316 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:00.039050 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:00.039302 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:00.039377 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:00.042045 (+ 2668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.131:59125: BlockingRecv error: recv got EOF from 127.5.96.131:59125 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 12 03:04:00 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:04:00 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:04:00.647395 20321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:00.642437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.129:50151)
0812 03:04:00.642747 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:00.642761 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:00.642861 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:00.643114 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:00.643116 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:00.643309 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:00.643412 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:00.643417 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:00.644121 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:00.644123 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:00.644682 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:00.644688 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:00.645059 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:00.645062 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:00.645121 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:00.645558 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:00.645568 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:00.647236 (+ 1668us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":69,"threads_started":1}
W20260812 03:04:00.647485 20322 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:00.642502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:50151 (local address 127.5.96.190:37713)
0812 03:04:00.642755 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:00.642759 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:00.642821 (+ 62us) server_negotiation.cc:407] Connection header received
0812 03:04:00.642940 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:00.642942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:00.643009 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:00.643075 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:00.643493 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:00.644050 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:00.644772 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:00.644985 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:00.645050 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:00.647373 (+ 2323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.129:50151: BlockingRecv error: recv got EOF from 127.5.96.129:50151 (error 108)
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":73,"threads_started":1}
Aug 12 03:04:00 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:04:00 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:04:00.829140 20323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:00.824077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.130:49207)
0812 03:04:00.824426 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:00.824439 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:00.824543 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:00.824791 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:00.824793 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:00.824979 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:00.825077 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:00.825083 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:00.825773 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:00.825775 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:00.826321 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:00.826328 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:00.826711 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:00.826714 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:00.826784 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:00.827286 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:00.827299 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:00.829022 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":99,"threads_started":1}
W20260812 03:04:00.829285 20322 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:00.824176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:49207 (local address 127.5.96.190:37713)
0812 03:04:00.824336 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:00.824341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:00.824504 (+ 163us) server_negotiation.cc:407] Connection header received
0812 03:04:00.824613 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:00.824615 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:00.824657 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:00.824722 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:00.825166 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:00.825686 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:00.826412 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:00.826627 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:00.826699 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:00.829153 (+ 2454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.130:49207: BlockingRecv error: recv got EOF from 127.5.96.130:49207 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 12 03:04:01 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:04:01 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:04:01.047863 20324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:01.042767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.131:57813)
0812 03:04:01.043070 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:01.043083 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:01.043190 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:01.043395 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:01.043397 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:01.043582 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:01.043712 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:01.043718 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:01.044390 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:01.044392 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:01.044978 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:01.044986 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:01.045349 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:01.045351 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:01.045410 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:01.045914 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:01.045927 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:01.047736 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":69,"threads_started":1}
W20260812 03:04:01.047978 20322 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:01.042818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:57813 (local address 127.5.96.190:37713)
0812 03:04:01.042940 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:01.042944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:01.043149 (+ 205us) server_negotiation.cc:407] Connection header received
0812 03:04:01.043261 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:01.043263 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:01.043307 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:01.043369 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:01.043794 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:01.044309 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:01.045063 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:01.045273 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:01.045338 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:01.047862 (+ 2524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.131:57813: BlockingRecv error: recv got EOF from 127.5.96.131:57813 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 12 03:04:01 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:04:01 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:04:01.653329 20326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:01.648095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.129:55829)
0812 03:04:01.648354 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:01.648367 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:01.648456 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:01.648661 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:01.648662 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:01.648855 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:01.648961 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:01.648966 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:01.649651 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:01.649653 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:01.650342 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:01.650356 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:01.650814 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:01.650817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:01.650896 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:01.651358 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:01.651371 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:01.653168 (+ 1797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":72,"threads_started":1}
W20260812 03:04:01.653424 20327 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:01.648222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:55829 (local address 127.5.96.190:37713)
0812 03:04:01.648450 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:01.648453 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:01.648465 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:04:01.648521 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:01.648523 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:01.648564 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:01.648637 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:01.649043 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:01.649574 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:01.650448 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:01.650663 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:01.650725 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:01.653308 (+ 2583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.129:55829: BlockingRecv error: recv got EOF from 127.5.96.129:55829 (error 108)
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":78,"threads_started":1}
Aug 12 03:04:01 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:04:01 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:04:01.835162 20328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:01.829862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.130:54017)
0812 03:04:01.830160 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:01.830175 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:01.830279 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:01.830487 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:01.830489 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:01.830684 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:01.830788 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:01.830794 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:01.831613 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:01.831615 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:01.832217 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:01.832226 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:01.832675 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:01.832678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:01.832777 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:01.833256 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:01.833268 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:01.835010 (+ 1742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":97,"threads_started":1}
W20260812 03:04:01.835242 20327 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:01.829926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:54017 (local address 127.5.96.190:37713)
0812 03:04:01.830076 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:01.830081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:01.830240 (+ 159us) server_negotiation.cc:407] Connection header received
0812 03:04:01.830351 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:01.830354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:01.830398 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:01.830460 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:01.830871 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:01.831520 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:01.832305 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:01.832557 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:01.832639 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:01.835143 (+ 2504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.130:54017: BlockingRecv error: recv got EOF from 127.5.96.130:54017 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 12 03:04:02 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:04:02 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:04:02.053498 20330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:02.048534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.131:44233)
0812 03:04:02.048805 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:02.048818 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:02.048912 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:02.049117 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:02.049119 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:02.049295 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:02.049391 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:02.049396 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:02.050046 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:02.050049 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:02.050633 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:02.050641 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:02.051023 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:02.051025 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:02.051084 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:02.051575 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:02.051590 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:02.053351 (+ 1761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":100,"threads_started":1}
W20260812 03:04:02.053584 20327 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:02.048592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:44233 (local address 127.5.96.190:37713)
0812 03:04:02.048718 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:02.048723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:02.048879 (+ 156us) server_negotiation.cc:407] Connection header received
0812 03:04:02.048982 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:02.048984 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:02.049030 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:02.049094 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:02.049467 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:02.049964 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:02.050724 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:02.050936 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:02.051005 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:02.053483 (+ 2478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.131:44233: BlockingRecv error: recv got EOF from 127.5.96.131:44233 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 12 03:04:02 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:04:02 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:04:02.663221 20337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:02.654029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.129:41101)
0812 03:04:02.654287 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:02.654303 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:02.654400 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:02.654597 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:02.654600 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:02.654784 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:02.654885 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:02.654890 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:02.655579 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:02.655582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:02.656186 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:02.656195 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:02.656567 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:02.656569 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:02.656628 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:02.661214 (+ 4586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:02.661228 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:02.663059 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":72,"threads_started":1}
W20260812 03:04:02.663298 20338 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:02.654163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:41101 (local address 127.5.96.190:37713)
0812 03:04:02.654402 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:02.654406 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:02.654419 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:04:02.654456 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:02.654458 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:02.654498 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:02.654570 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:02.654971 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:02.655502 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:02.656274 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:02.656493 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:02.656556 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:02.663187 (+ 6631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.129:41101: BlockingRecv error: recv got EOF from 127.5.96.129:41101 (error 108)
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":74,"threads_started":1}
Aug 12 03:04:02 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:04:02 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:04:02.840914 20340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:02.835835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.130:51599)
0812 03:04:02.836081 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:02.836094 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:02.836192 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:02.836390 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:02.836392 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:02.836569 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:02.836663 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:02.836668 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:02.837337 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:02.837339 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:02.837882 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:02.837888 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:02.838234 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:02.838236 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:02.838296 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:02.838740 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:02.838751 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:02.840781 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":77,"threads_started":1}
W20260812 03:04:02.841007 20338 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:02.835894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:51599 (local address 127.5.96.190:37713)
0812 03:04:02.836050 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:02.836055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:02.836153 (+ 98us) server_negotiation.cc:407] Connection header received
0812 03:04:02.836257 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:02.836259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:02.836303 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:02.836366 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:02.836752 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:02.837261 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:02.837957 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:02.838162 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:02.838225 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:02.840910 (+ 2685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.130:51599: BlockingRecv error: recv got EOF from 127.5.96.130:51599 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 03:04:03 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:04:03 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:04:03.059104 20344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:03.054136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.131:40463)
0812 03:04:03.054382 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:03.054396 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:03.054493 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:03.054698 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:03.054700 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:03.054882 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:03.054979 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:03.054984 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:03.055636 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:03.055638 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:03.056282 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:03.056292 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:03.056664 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:03.056666 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:03.056721 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:03.057149 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:03.057160 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:03.058928 (+ 1768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":73,"threads_started":1}
W20260812 03:04:03.059161 20338 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:03.054191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:40463 (local address 127.5.96.190:37713)
0812 03:04:03.054315 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:03.054319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:03.054453 (+ 134us) server_negotiation.cc:407] Connection header received
0812 03:04:03.054574 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:03.054576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:03.054620 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:03.054676 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:03.055055 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:03.055559 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:03.056370 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:03.056583 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:03.056655 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:03.059063 (+ 2408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.131:40463: BlockingRecv error: recv got EOF from 127.5.96.131:40463 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 12 03:04:03 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:04:03 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:04:03.669252 20345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:03.663969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.129:43901)
0812 03:04:03.664231 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:03.664243 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:03.664345 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:03.664545 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:03.664547 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:03.664723 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:03.664829 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:03.664834 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:03.665507 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:03.665510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:03.666302 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:03.666315 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:03.666867 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:03.666872 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:03.666961 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:03.667392 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:03.667402 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:03.669125 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":82,"threads_started":1}
W20260812 03:04:03.669384 20346 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:03.664040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:43901 (local address 127.5.96.190:37713)
0812 03:04:03.664261 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:03.664265 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:03.664313 (+ 48us) server_negotiation.cc:407] Connection header received
0812 03:04:03.664412 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:03.664414 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:03.664454 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:03.664524 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:03.664909 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:03.665432 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:03.666418 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:03.666738 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:03.666845 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:03.669253 (+ 2408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.129:43901: BlockingRecv error: recv got EOF from 127.5.96.129:43901 (error 108)
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":60,"threads_started":1}
Aug 12 03:04:03 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:04:03 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:04:03.846423 20348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:03.841638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.130:44311)
0812 03:04:03.841881 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:03.841894 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:03.841987 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:03.842201 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:03.842204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:03.842379 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:03.842474 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:03.842479 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:03.843156 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:03.843158 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:03.843758 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:03.843768 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:03.844116 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:03.844118 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:03.844175 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:03.844617 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:03.844627 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:03.846306 (+ 1679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":72,"threads_started":1}
W20260812 03:04:03.846544 20346 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:03.841699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:44311 (local address 127.5.96.190:37713)
0812 03:04:03.841863 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:03.841867 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:03.841956 (+ 89us) server_negotiation.cc:407] Connection header received
0812 03:04:03.842054 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:03.842056 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:03.842108 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:03.842172 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:03.842554 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:03.843077 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:03.843840 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:03.844044 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:03.844110 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:03.846428 (+ 2318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.130:44311: BlockingRecv error: recv got EOF from 127.5.96.130:44311 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 12 03:04:04 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:04:04 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:04:04.064832 20349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:04.059922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.131:44663)
0812 03:04:04.060180 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:04.060193 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:04.060285 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:04.060484 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:04.060487 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:04.060660 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:04.060769 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:04.060774 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:04.061436 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:04.061439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:04.061991 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:04.061998 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:04.062365 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:04.062367 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:04.062423 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:04.062902 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:04.062914 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:04.064663 (+ 1749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":78,"threads_started":1}
W20260812 03:04:04.064898 20346 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:04.060032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:44663 (local address 127.5.96.190:37713)
0812 03:04:04.060155 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:04.060158 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:04.060255 (+ 97us) server_negotiation.cc:407] Connection header received
0812 03:04:04.060359 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:04.060361 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:04.060404 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:04.060459 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:04.060848 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:04.061359 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:04.062066 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:04.062288 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:04.062355 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:04.064797 (+ 2442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.131:44663: BlockingRecv error: recv got EOF from 127.5.96.131:44663 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 12 03:04:04 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:04:04 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:04:04.675117 20350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:04.669959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.129:40441)
0812 03:04:04.670233 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:04.670255 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:04.670354 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:04.670582 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:04.670584 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:04.670767 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:04.670872 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:04.670877 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:04.671566 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:04.671568 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:04.672141 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:04.672148 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:04.672532 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:04.672534 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:04.672591 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:04.673071 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:04.673089 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:04.674938 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":114,"threads_started":1}
W20260812 03:04:04.675201 20351 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:04.670033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:40441 (local address 127.5.96.190:37713)
0812 03:04:04.670259 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:04.670262 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:04.670318 (+ 56us) server_negotiation.cc:407] Connection header received
0812 03:04:04.670426 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:04.670428 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:04.670471 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:04.670545 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:04.670956 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:04.671486 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:04.672226 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:04.672449 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:04.672523 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:04.675084 (+ 2561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.129:40441: BlockingRecv error: recv got EOF from 127.5.96.129:40441 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":61,"threads_started":1}
Aug 12 03:04:04 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:04:04 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:04:04.852730 20352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:04.847065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.130:42801)
0812 03:04:04.847344 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:04.847358 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:04.847467 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:04.847726 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:04.847729 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:04.847949 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:04.848048 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:04.848054 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:04.848793 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:04.848797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:04.849348 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:04.849356 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:04.849786 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:04.849790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:04.849873 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:04.850450 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:04.850463 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:04.852608 (+ 2145us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":86,"threads_started":1}
W20260812 03:04:04.852859 20351 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:04.847122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:42801 (local address 127.5.96.190:37713)
0812 03:04:04.847267 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:04.847272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:04.847423 (+ 151us) server_negotiation.cc:407] Connection header received
0812 03:04:04.847559 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:04.847562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:04.847610 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:04.847706 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:04.848155 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:04.848687 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:04.849456 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:04.849681 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:04.849765 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:04.852724 (+ 2959us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.130:42801: BlockingRecv error: recv got EOF from 127.5.96.130:42801 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 03:04:05 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:04:05 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:04:05.070439 20355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:05.065575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.131:51241)
0812 03:04:05.065844 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:05.065857 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:05.065953 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:05.066150 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:05.066153 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:05.066326 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:05.066420 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:05.066425 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:05.067089 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:05.067091 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:05.067629 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:05.067635 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:05.068045 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:05.068047 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:05.068109 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:05.068663 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:05.068680 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:05.070327 (+ 1647us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":97,"threads_started":1}
W20260812 03:04:05.070570 20351 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:05.065630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:51241 (local address 127.5.96.190:37713)
0812 03:04:05.065779 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:05.065784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:05.065918 (+ 134us) server_negotiation.cc:407] Connection header received
0812 03:04:05.066022 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:05.066024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:05.066068 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:05.066128 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:05.066497 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:05.067013 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:05.067759 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:05.067964 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:05.068026 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:05.070452 (+ 2426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.131:51241: BlockingRecv error: recv got EOF from 127.5.96.131:51241 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 03:04:05 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:04:05 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:04:05.680995 20360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:05.675930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.129:60717)
0812 03:04:05.676244 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:05.676258 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:05.676354 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:05.676554 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:05.676556 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:05.676737 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:05.676839 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:05.676844 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:05.677518 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:05.677520 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:05.678070 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:05.678077 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:05.678468 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:05.678471 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:05.678540 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:05.679031 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:05.679042 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:05.680844 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":91,"threads_started":1}
W20260812 03:04:05.681124 20363 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:05.675996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:60717 (local address 127.5.96.190:37713)
0812 03:04:05.676246 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:05.676249 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:05.676306 (+ 57us) server_negotiation.cc:407] Connection header received
0812 03:04:05.676416 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:05.676418 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:05.676459 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:05.676529 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:05.676920 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:05.677438 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:05.678170 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:05.678377 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:05.678439 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:05.680982 (+ 2543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.129:60717: BlockingRecv error: recv got EOF from 127.5.96.129:60717 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":74,"threads_started":1}
Aug 12 03:04:05 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:04:05 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:04:05.858708 20364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:05.853503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.130:43209)
0812 03:04:05.853799 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:05.853813 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:05.853911 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:05.854116 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:05.854118 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:05.854305 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:05.854402 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:05.854407 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:05.855210 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:05.855213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:05.855797 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:05.855806 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:05.856239 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:05.856241 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:05.856294 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:05.856758 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:05.856770 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:05.858558 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":102,"threads_started":1}
W20260812 03:04:05.858846 20363 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:05.853568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:43209 (local address 127.5.96.190:37713)
0812 03:04:05.853696 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:05.853700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:05.853871 (+ 171us) server_negotiation.cc:407] Connection header received
0812 03:04:05.853978 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:05.853980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:05.854024 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:05.854091 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:05.854574 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:05.855116 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:05.855894 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:05.856159 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:05.856234 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:05.858735 (+ 2501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.130:43209: BlockingRecv error: recv got EOF from 127.5.96.130:43209 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 03:04:06 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:04:06 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:04:06.076812 20365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:06.071172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.131:34545)
0812 03:04:06.071446 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:06.071459 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:06.071559 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:06.071820 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:06.071822 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:06.072005 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:06.072110 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:06.072116 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:06.073078 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:06.073082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:06.073834 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:06.073844 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:06.074287 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:06.074289 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:06.074377 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:06.074844 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:06.074855 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:06.076694 (+ 1839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":95,"threads_started":1}
W20260812 03:04:06.076938 20363 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:06.071208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:34545 (local address 127.5.96.190:37713)
0812 03:04:06.071345 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:06.071348 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:06.071520 (+ 172us) server_negotiation.cc:407] Connection header received
0812 03:04:06.071641 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:06.071643 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:06.071707 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:06.071792 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:06.072193 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:06.072956 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:06.073950 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:06.074194 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:06.074289 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:06.076831 (+ 2542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.131:34545: BlockingRecv error: recv got EOF from 127.5.96.131:34545 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 12 03:04:06 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:04:06 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:04:06.686749 20367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:06.681736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.129:42389)
0812 03:04:06.682016 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:06.682028 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:06.682125 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:06.682337 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:06.682339 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:06.682519 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:06.682619 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:06.682624 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:06.683320 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:06.683323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:06.683955 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:06.683965 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:06.684330 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:06.684332 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:06.684390 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:06.684843 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:06.684854 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:06.686587 (+ 1733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":83,"threads_started":1}
W20260812 03:04:06.686851 20368 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:06.681807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:42389 (local address 127.5.96.190:37713)
0812 03:04:06.682053 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:06.682057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:06.682085 (+ 28us) server_negotiation.cc:407] Connection header received
0812 03:04:06.682196 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:06.682198 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:06.682239 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:06.682315 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:06.682702 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:06.683216 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:06.684039 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:06.684258 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:06.684322 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:06.686721 (+ 2399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.129:42389: BlockingRecv error: recv got EOF from 127.5.96.129:42389 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":79,"threads_started":1}
Aug 12 03:04:06 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:04:06 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:04:06.864423 20369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:06.859440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.130:47951)
0812 03:04:06.859731 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:06.859744 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:06.859846 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:06.860048 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:06.860050 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:06.860230 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:06.860327 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:06.860332 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:06.860980 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:06.860983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:06.861546 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:06.861554 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:06.861898 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:06.861900 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:06.861956 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:06.862470 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:06.862482 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:06.864259 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":81,"threads_started":1}
W20260812 03:04:06.864529 20368 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:06.859572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:47951 (local address 127.5.96.190:37713)
0812 03:04:06.859731 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:06.859735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:06.859800 (+ 65us) server_negotiation.cc:407] Connection header received
0812 03:04:06.859915 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:06.859918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:06.859960 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:06.860024 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:06.860407 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:06.860909 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:06.861622 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:06.861826 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:06.861890 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:06.864428 (+ 2538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.130:47951: BlockingRecv error: recv got EOF from 127.5.96.130:47951 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 12 03:04:07 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:04:07 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:04:07.083186 20370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:07.077532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.131:51239)
0812 03:04:07.077927 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:07.077942 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:07.078054 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:07.078304 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:07.078308 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:07.078537 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:07.078643 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:07.078650 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:07.079520 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:07.079524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:07.080351 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:07.080364 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:07.080821 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:07.080825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:07.080905 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:07.081333 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:07.081350 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:07.083060 (+ 1710us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":99,"threads_started":1}
W20260812 03:04:07.083313 20368 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:07.077658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:51239 (local address 127.5.96.190:37713)
0812 03:04:07.077787 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:07.077791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:07.078012 (+ 221us) server_negotiation.cc:407] Connection header received
0812 03:04:07.078141 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:07.078143 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:07.078189 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:07.078257 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:07.078766 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:07.079409 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:07.080468 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:07.080710 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:07.080805 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:07.083197 (+ 2392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.131:51239: BlockingRecv error: recv got EOF from 127.5.96.131:51239 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 03:04:07 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:04:07 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:04:07.700747 20372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:07.687458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.129:46481)
0812 03:04:07.687764 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:07.687777 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:07.687867 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:07.688071 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:07.688073 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:07.688254 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:07.688351 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:07.688355 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:07.689055 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:07.689057 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:07.689631 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:07.689638 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:07.690106 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:07.690111 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:07.690202 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:07.698790 (+ 8588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:07.698803 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:07.700612 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":76,"threads_started":1}
W20260812 03:04:07.700884 20374 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:07.687539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:46481 (local address 127.5.96.190:37713)
0812 03:04:07.687861 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:07.687865 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:07.687878 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:04:07.687929 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:07.687931 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:07.687972 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:07.688047 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:07.688435 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:07.688971 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:07.689738 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:07.689994 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:07.690128 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:07.700752 (+ 10624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.129:46481: BlockingRecv error: recv got EOF from 127.5.96.129:46481 (error 108)
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":97,"threads_started":1}
Aug 12 03:04:07 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:04:07 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:04:07.870335 20375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:07.865294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.130:47451)
0812 03:04:07.865549 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:07.865562 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:07.865659 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:07.865860 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:07.865862 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:07.866040 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:07.866136 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:07.866141 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:07.866818 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:07.866820 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:07.867392 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:07.867399 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:07.867810 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:07.867813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:07.867870 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:07.868395 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:07.868407 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:07.870172 (+ 1765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":78,"threads_started":1}
W20260812 03:04:07.870416 20374 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:07.865366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:47451 (local address 127.5.96.190:37713)
0812 03:04:07.865511 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:07.865515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:07.865622 (+ 107us) server_negotiation.cc:407] Connection header received
0812 03:04:07.865724 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:07.865726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:07.865771 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:07.865834 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:07.866215 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:07.866737 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:07.867488 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:07.867725 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:07.867804 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:07.870317 (+ 2513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.130:47451: BlockingRecv error: recv got EOF from 127.5.96.130:47451 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 03:04:08 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:04:08 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.131@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:04:08.088716 20376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:08.083841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.131:46813)
0812 03:04:08.084089 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:08.084102 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:08.084201 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:08.084411 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:08.084414 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:08.084592 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:08.084687 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:08.084692 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:08.085367 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:08.085370 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:08.085941 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:08.085949 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:08.086301 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:08.086304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:08.086361 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:08.086780 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:08.086791 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:08.088551 (+ 1760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":74,"threads_started":1}
W20260812 03:04:08.088784 20374 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:08.083896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:46813 (local address 127.5.96.190:37713)
0812 03:04:08.084041 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:08.084045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:08.084162 (+ 117us) server_negotiation.cc:407] Connection header received
0812 03:04:08.084273 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:08.084275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:08.084318 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:08.084380 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:08.084774 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:08.085291 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:08.086019 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:08.086221 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:08.086293 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:08.088684 (+ 2391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.131:46813: BlockingRecv error: recv got EOF from 127.5.96.131:46813 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 12 03:04:08 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:04:08 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.129@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:04:08.706640 20378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:08.701521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.129:45483)
0812 03:04:08.701779 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:08.701792 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:08.701877 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:08.702084 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:08.702096 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:08.702288 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:08.702391 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:08.702396 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:08.703084 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:08.703086 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:08.703688 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:08.703698 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:08.704166 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:08.704170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:08.704272 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:08.704792 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:08.704804 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:08.706479 (+ 1675us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":68,"threads_started":1}
W20260812 03:04:08.706728 20380 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:08.701578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:45483 (local address 127.5.96.190:37713)
0812 03:04:08.701840 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:08.701843 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:08.701857 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:04:08.701950 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:08.701952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:08.701992 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:08.702061 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:08.702474 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:08.703006 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:08.703775 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:08.704040 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:08.704147 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:08.706615 (+ 2468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.129:45483: BlockingRecv error: recv got EOF from 127.5.96.129:45483 (error 108)
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":77,"threads_started":1}
Aug 12 03:04:08 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:04:08 dist-test-slave-6bbx krb5kdc[18455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.96.130@KRBTEST.COM for oryx/127.5.96.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:04:08.876454 20381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:08.871244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37713 (local address 127.5.96.130:54103)
0812 03:04:08.871531 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:08.871543 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:08.871644 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:08.871910 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:08.871913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:08.872119 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:08.872213 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:08.872218 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:08.872956 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:08.872960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:08.873522 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:08.873530 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:08.873962 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:08.873965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:08.874048 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:08.874554 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:08.874566 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:08.876306 (+ 1740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.96.190:37713: Server oryx/127.5.96.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":96,"threads_started":1}
W20260812 03:04:08.876544 20380 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:08.871319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:54103 (local address 127.5.96.190:37713)
0812 03:04:08.871464 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:08.871468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:08.871609 (+ 141us) server_negotiation.cc:407] Connection header received
0812 03:04:08.871762 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:08.871764 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:08.871810 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:08.871875 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:08.872331 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:08.872854 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:08.873633 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:08.873862 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:08.873946 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:08.876444 (+ 2498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.96.130:54103: BlockingRecv error: recv got EOF from 127.5.96.130:54103 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:04:09.005538 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 19484
I20260812 03:04:09.023343 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 19787
I20260812 03:04:09.038385 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 20030
I20260812 03:04:09.052729 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 19355
2026-08-12T03:04:09Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18344 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T03:04:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:04:09Z Disabled control of system clock
I20260812 03:04:09.087015 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:38553
--webserver_interface=127.5.96.190
--webserver_port=0
--builtin_ntp_servers=127.5.96.148:43039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:38553
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 03:04:09.194062 20386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:04:09.194303 20386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:04:09.194336 20386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:04:09.198207 20386 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:04:09.198259 20386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:04:09.198279 20386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:04:09.198297 20386 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:04:09.198313 20386 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:04:09.202819 20386 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:43039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:38553
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:38553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:04:09.203859 20386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:04:09.205482 20386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:04:09.213480 20397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:09.213480 20392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:09.213482 20393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:04:09.213804 20386 server_base.cc:1034] running on GCE node
I20260812 03:04:09.214339 20386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:04:09.215358 20386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:04:09.216514 20386 hybrid_clock.cc:648] HybridClock initialized: now 1786503849216494 us; error 28 us; skew 500 ppm
I20260812 03:04:09.218189 20386 webserver.cc:460] Webserver started at http://127.5.96.190:36695/ using document root <none> and password file <none>
I20260812 03:04:09.218704 20386 fs_manager.cc:359] Metadata directory not provided
I20260812 03:04:09.218778 20386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:04:09.218977 20386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:04:09.220939 20386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "76c43ce116d04430939869c2e2ca7e34"
format_stamp: "Formatted at 2026-08-12 03:04:09 on dist-test-slave-6bbx"
server_key: "0acffb862aa98589b2ca94408f0a0ad7"
server_key_iv: "1f253b252c44610e796a6228dfa9168d"
server_key_version: "encryptionkey@0"
I20260812 03:04:09.221455 20386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "76c43ce116d04430939869c2e2ca7e34"
format_stamp: "Formatted at 2026-08-12 03:04:09 on dist-test-slave-6bbx"
server_key: "0acffb862aa98589b2ca94408f0a0ad7"
server_key_iv: "1f253b252c44610e796a6228dfa9168d"
server_key_version: "encryptionkey@0"
I20260812 03:04:09.224864 20386 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:04:09.227026 20408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:09.228261 20386 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:04:09.228374 20386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "76c43ce116d04430939869c2e2ca7e34"
format_stamp: "Formatted at 2026-08-12 03:04:09 on dist-test-slave-6bbx"
server_key: "0acffb862aa98589b2ca94408f0a0ad7"
server_key_iv: "1f253b252c44610e796a6228dfa9168d"
server_key_version: "encryptionkey@0"
I20260812 03:04:09.228461 20386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:04:09.242707 20386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:04:09.243256 20386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:04:09.243412 20386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:04:09.249805 20386 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:38553
I20260812 03:04:09.249811 20506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:38553 every 8 connection(s)
I20260812 03:04:09.250664 20386 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:04:09.252059 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 20386
I20260812 03:04:09.252177 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:04:09.252494 5506 external_mini_cluster.cc:1428] Setting key 20e5d1ac0083afa398e0be6aa52020fd
I20260812 03:04:09.253742 20507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:04:09.259032 20507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76c43ce116d04430939869c2e2ca7e34: Bootstrap starting.
I20260812 03:04:09.260699 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:09.253758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33938 (local address 127.5.96.190:38553)
0812 03:04:09.254341 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:09.254355 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:09.254387 (+ 32us) server_negotiation.cc:407] Connection header received
0812 03:04:09.254998 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:09.255014 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:09.255369 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:09.255648 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:09.256059 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:09.256975 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:09.257827 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:09.258263 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:09.258400 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:09.259166 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:04:09.259172 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:04:09.259187 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:04:09.259528 (+ 341us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:04:09.259580 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:04:09.259600 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:04:09.259707 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:09.259841 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 03:04:09.260069 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":140,"threads_started":1}
I20260812 03:04:09.261250 20507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76c43ce116d04430939869c2e2ca7e34: Neither blocks nor log segments found. Creating new log.
I20260812 03:04:09.262019 20507 log.cc:824] T 00000000000000000000000000000000 P 76c43ce116d04430939869c2e2ca7e34: Log is configured to *not* fsync() on all Append() calls
I20260812 03:04:09.263953 20507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76c43ce116d04430939869c2e2ca7e34: No bootstrap required, opened a new log
I20260812 03:04:09.267066 20507 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76c43ce116d04430939869c2e2ca7e34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c43ce116d04430939869c2e2ca7e34" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 38553 } }
I20260812 03:04:09.267266 20507 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76c43ce116d04430939869c2e2ca7e34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:04:09.267306 20507 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76c43ce116d04430939869c2e2ca7e34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76c43ce116d04430939869c2e2ca7e34, State: Initialized, Role: FOLLOWER
I20260812 03:04:09.267632 20507 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76c43ce116d04430939869c2e2ca7e34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c43ce116d04430939869c2e2ca7e34" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 38553 } }
I20260812 03:04:09.267781 20507 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76c43ce116d04430939869c2e2ca7e34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:04:09.267828 20507 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76c43ce116d04430939869c2e2ca7e34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:04:09.267899 20507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76c43ce116d04430939869c2e2ca7e34 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:04:09.268756 20507 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76c43ce116d04430939869c2e2ca7e34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c43ce116d04430939869c2e2ca7e34" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 38553 } }
I20260812 03:04:09.269039 20507 leader_election.cc:302] T 00000000000000000000000000000000 P 76c43ce116d04430939869c2e2ca7e34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76c43ce116d04430939869c2e2ca7e34; no voters:
I20260812 03:04:09.269266 20507 leader_election.cc:288] T 00000000000000000000000000000000 P 76c43ce116d04430939869c2e2ca7e34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:04:09.269380 20518 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76c43ce116d04430939869c2e2ca7e34 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:04:09.269575 20518 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76c43ce116d04430939869c2e2ca7e34 [term 1 LEADER]: Becoming Leader. State: Replica: 76c43ce116d04430939869c2e2ca7e34, State: Running, Role: LEADER
I20260812 03:04:09.269917 20518 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76c43ce116d04430939869c2e2ca7e34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c43ce116d04430939869c2e2ca7e34" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 38553 } }
I20260812 03:04:09.270287 20507 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76c43ce116d04430939869c2e2ca7e34 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:04:09.271347 20524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76c43ce116d04430939869c2e2ca7e34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76c43ce116d04430939869c2e2ca7e34. Latest consensus state: current_term: 1 leader_uuid: "76c43ce116d04430939869c2e2ca7e34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c43ce116d04430939869c2e2ca7e34" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 38553 } } }
I20260812 03:04:09.271473 20524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76c43ce116d04430939869c2e2ca7e34 [sys.catalog]: This master's current role is: LEADER
I20260812 03:04:09.271910 20538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:04:09.271883 20523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76c43ce116d04430939869c2e2ca7e34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76c43ce116d04430939869c2e2ca7e34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c43ce116d04430939869c2e2ca7e34" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 38553 } } }
I20260812 03:04:09.272099 20523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76c43ce116d04430939869c2e2ca7e34 [sys.catalog]: This master's current role is: LEADER
I20260812 03:04:09.274499 20538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:04:09.279253 20538 catalog_manager.cc:1347] Generated new cluster ID: 3c3d005a6acb4bb59bf33ba8620c7907
I20260812 03:04:09.279312 20538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:04:09.290591 20538 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:04:09.291347 20538 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:04:09.304368 20538 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76c43ce116d04430939869c2e2ca7e34: Generated new TSK 0
I20260812 03:04:09.305042 20538 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:04:09.309113 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:0
--local_ip_for_outbound_sockets=127.5.96.129
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:38553
--builtin_ntp_servers=127.5.96.148:43039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 03:04:09.418232 20562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:04:09.418449 20562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:04:09.418483 20562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:04:09.422336 20562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:04:09.422379 20562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:04:09.422425 20562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:04:09.426884 20562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:43039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:38553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:04:09.427783 20562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:04:09.429018 20562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:04:09.438066 20574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:09.438081 20573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:09.438074 20580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:04:09.438220 20562 server_base.cc:1034] running on GCE node
I20260812 03:04:09.438738 20562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:04:09.439301 20562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:04:09.440451 20562 hybrid_clock.cc:648] HybridClock initialized: now 1786503849440435 us; error 25 us; skew 500 ppm
I20260812 03:04:09.442150 20562 webserver.cc:460] Webserver started at http://127.5.96.129:36477/ using document root <none> and password file <none>
I20260812 03:04:09.442682 20562 fs_manager.cc:359] Metadata directory not provided
I20260812 03:04:09.442752 20562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:04:09.442955 20562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:04:09.444890 20562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/data/instance:
uuid: "cb7d3699b4a445b0b3b3472d8113f980"
format_stamp: "Formatted at 2026-08-12 03:04:09 on dist-test-slave-6bbx"
server_key: "55c0db857aca32acbf2fd694cad2d39e"
server_key_iv: "9dc43f94ee163311da3f6ae419ec7521"
server_key_version: "encryptionkey@0"
I20260812 03:04:09.445418 20562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance:
uuid: "cb7d3699b4a445b0b3b3472d8113f980"
format_stamp: "Formatted at 2026-08-12 03:04:09 on dist-test-slave-6bbx"
server_key: "55c0db857aca32acbf2fd694cad2d39e"
server_key_iv: "9dc43f94ee163311da3f6ae419ec7521"
server_key_version: "encryptionkey@0"
I20260812 03:04:09.448802 20562 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:04:09.451009 20589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:09.452144 20562 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:04:09.452245 20562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "cb7d3699b4a445b0b3b3472d8113f980"
format_stamp: "Formatted at 2026-08-12 03:04:09 on dist-test-slave-6bbx"
server_key: "55c0db857aca32acbf2fd694cad2d39e"
server_key_iv: "9dc43f94ee163311da3f6ae419ec7521"
server_key_version: "encryptionkey@0"
I20260812 03:04:09.452338 20562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:04:09.481091 20562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:04:09.481654 20562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:04:09.481817 20562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:04:09.482347 20562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:04:09.483212 20562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:04:09.483271 20562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:09.483322 20562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:04:09.483348 20562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:09.491317 20562 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:36059
I20260812 03:04:09.491338 20805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:36059 every 8 connection(s)
I20260812 03:04:09.492128 20562 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:04:09.495711 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 20562
I20260812 03:04:09.495800 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance
I20260812 03:04:09.496058 5506 external_mini_cluster.cc:1428] Setting key 7feaf1af50e018869505fcbee0f8f9b4
I20260812 03:04:09.497825 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:0
--local_ip_for_outbound_sockets=127.5.96.130
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:38553
--builtin_ntp_servers=127.5.96.148:43039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 03:04:09.500921 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:09.493751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:47657 (local address 127.5.96.190:38553)
0812 03:04:09.493889 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:09.493893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:09.494909 (+ 1016us) server_negotiation.cc:407] Connection header received
0812 03:04:09.496340 (+ 1431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:09.496344 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:09.496405 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:09.496476 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:09.497470 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:09.498044 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:09.499121 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:09.499353 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:09.499441 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:09.500462 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:04:09.500466 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:04:09.500469 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:04:09.500482 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:04:09.500499 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:04:09.500507 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:04:09.500565 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:09.500705 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 03:04:09.500780 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:04:09.501699 20810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:09.494141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:38553 (local address 127.5.96.129:47657)
0812 03:04:09.494775 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:09.494814 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:09.496130 (+ 1316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:09.496664 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:09.496673 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:09.496771 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:09.497342 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:09.497357 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:09.498161 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:09.498165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:09.499008 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:09.499018 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:09.499491 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:09.499495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:09.499644 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:09.500227 (+ 583us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:04:09.500229 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:09.500267 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:04:09.500326 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:04:09.500575 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:04:09.500585 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:04:09.500591 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:04:09.500697 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 03:04:09.500958 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":112,"threads_started":1}
I20260812 03:04:09.502782 20807 heartbeater.cc:347] Connected to a master server at 127.5.96.190:38553
I20260812 03:04:09.503010 20807 heartbeater.cc:464] Registering TS with master...
I20260812 03:04:09.503629 20807 heartbeater.cc:511] Master 127.5.96.190:38553 requested a full tablet report, sending...
I20260812 03:04:09.505105 20438 ts_manager.cc:194] Registered new tserver with Master: cb7d3699b4a445b0b3b3472d8113f980 (127.5.96.129:36059)
I20260812 03:04:09.506189 20438 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.96.129:47657
W20260812 03:04:09.605903 20812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:04:09.606115 20812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:04:09.606149 20812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:04:09.609989 20812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:04:09.610031 20812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:04:09.610092 20812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:04:09.614635 20812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:43039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:38553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:04:09.615507 20812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:04:09.616720 20812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:04:09.625385 20829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:09.625427 20822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:09.625401 20825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:04:09.625625 20812 server_base.cc:1034] running on GCE node
I20260812 03:04:09.626008 20812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:04:09.626550 20812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:04:09.627720 20812 hybrid_clock.cc:648] HybridClock initialized: now 1786503849627694 us; error 35 us; skew 500 ppm
I20260812 03:04:09.629359 20812 webserver.cc:460] Webserver started at http://127.5.96.130:38115/ using document root <none> and password file <none>
I20260812 03:04:09.629861 20812 fs_manager.cc:359] Metadata directory not provided
I20260812 03:04:09.629926 20812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:04:09.630100 20812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:04:09.631975 20812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/data/instance:
uuid: "bc4b22d16ee1490b819144b2e611497f"
format_stamp: "Formatted at 2026-08-12 03:04:09 on dist-test-slave-6bbx"
server_key: "0564a688b39e925ec31ecd5f181cf0dc"
server_key_iv: "7f04130e56650490fc69ece8084af856"
server_key_version: "encryptionkey@0"
I20260812 03:04:09.632472 20812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance:
uuid: "bc4b22d16ee1490b819144b2e611497f"
format_stamp: "Formatted at 2026-08-12 03:04:09 on dist-test-slave-6bbx"
server_key: "0564a688b39e925ec31ecd5f181cf0dc"
server_key_iv: "7f04130e56650490fc69ece8084af856"
server_key_version: "encryptionkey@0"
I20260812 03:04:09.635687 20812 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:04:09.637857 20840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:09.638907 20812 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:04:09.639007 20812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "bc4b22d16ee1490b819144b2e611497f"
format_stamp: "Formatted at 2026-08-12 03:04:09 on dist-test-slave-6bbx"
server_key: "0564a688b39e925ec31ecd5f181cf0dc"
server_key_iv: "7f04130e56650490fc69ece8084af856"
server_key_version: "encryptionkey@0"
I20260812 03:04:09.639093 20812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:04:09.662153 20812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:04:09.662698 20812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:04:09.662847 20812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:04:09.663347 20812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:04:09.664201 20812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:04:09.664247 20812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:09.664285 20812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:04:09.664315 20812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:09.672324 20812 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:42017
I20260812 03:04:09.672358 21060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:42017 every 8 connection(s)
I20260812 03:04:09.673127 20812 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:04:09.675400 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 20812
I20260812 03:04:09.675513 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance
I20260812 03:04:09.675840 5506 external_mini_cluster.cc:1428] Setting key 2f4e8ca299b4b874e934e7753236daf6
I20260812 03:04:09.677811 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:0
--local_ip_for_outbound_sockets=127.5.96.131
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:38553
--builtin_ntp_servers=127.5.96.148:43039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 03:04:09.681988 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:09.674696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:52051 (local address 127.5.96.190:38553)
0812 03:04:09.674824 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:09.674828 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:09.675944 (+ 1116us) server_negotiation.cc:407] Connection header received
0812 03:04:09.677377 (+ 1433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:09.677380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:09.677424 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:09.677481 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:09.678516 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:09.679028 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:09.680290 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:09.680506 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:09.680595 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:09.681570 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:04:09.681573 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:04:09.681576 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:04:09.681587 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:04:09.681611 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:04:09.681619 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:04:09.681677 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:09.681827 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 03:04:09.681880 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:04:09.683087 21064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:09.675026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:38553 (local address 127.5.96.130:52051)
0812 03:04:09.675774 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:09.675844 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:09.677156 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:09.677669 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:09.677677 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:09.677786 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:09.678371 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:09.678389 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:09.679149 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:09.679153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:09.680171 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:09.680181 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:09.680644 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:09.680648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:09.680797 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:09.681297 (+ 500us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:04:09.681299 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:09.681346 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:04:09.681417 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:04:09.681694 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:04:09.681703 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:04:09.681709 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:04:09.681820 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 03:04:09.682163 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":112,"threads_started":1}
I20260812 03:04:09.684155 21061 heartbeater.cc:347] Connected to a master server at 127.5.96.190:38553
I20260812 03:04:09.684388 21061 heartbeater.cc:464] Registering TS with master...
I20260812 03:04:09.684937 21061 heartbeater.cc:511] Master 127.5.96.190:38553 requested a full tablet report, sending...
I20260812 03:04:09.685947 20438 ts_manager.cc:194] Registered new tserver with Master: bc4b22d16ee1490b819144b2e611497f (127.5.96.130:42017)
I20260812 03:04:09.686462 20438 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.96.130:52051
W20260812 03:04:09.787528 21065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:04:09.787781 21065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:04:09.787834 21065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:04:09.791654 21065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:04:09.791718 21065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:04:09.791788 21065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:04:09.796298 21065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:43039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:38553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:04:09.797163 21065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:04:09.798353 21065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:04:09.807283 21080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:09.807293 21085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:09.807297 21081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:04:09.807386 21065 server_base.cc:1034] running on GCE node
I20260812 03:04:09.808017 21065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:04:09.808575 21065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:04:09.809731 21065 hybrid_clock.cc:648] HybridClock initialized: now 1786503849809705 us; error 35 us; skew 500 ppm
I20260812 03:04:09.811373 21065 webserver.cc:460] Webserver started at http://127.5.96.131:45825/ using document root <none> and password file <none>
I20260812 03:04:09.811914 21065 fs_manager.cc:359] Metadata directory not provided
I20260812 03:04:09.811977 21065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:04:09.812170 21065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:04:09.814030 21065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/data/instance:
uuid: "ccc4b6ca5e7c408fb63883702398690f"
format_stamp: "Formatted at 2026-08-12 03:04:09 on dist-test-slave-6bbx"
server_key: "393a0bf2b9c36bdb3232b962fff54b1f"
server_key_iv: "c309eb33a78608f1215db7aed68f9580"
server_key_version: "encryptionkey@0"
I20260812 03:04:09.814550 21065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance:
uuid: "ccc4b6ca5e7c408fb63883702398690f"
format_stamp: "Formatted at 2026-08-12 03:04:09 on dist-test-slave-6bbx"
server_key: "393a0bf2b9c36bdb3232b962fff54b1f"
server_key_iv: "c309eb33a78608f1215db7aed68f9580"
server_key_version: "encryptionkey@0"
I20260812 03:04:09.817894 21065 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 03:04:09.820122 21098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:09.821177 21065 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:04:09.821288 21065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "ccc4b6ca5e7c408fb63883702398690f"
format_stamp: "Formatted at 2026-08-12 03:04:09 on dist-test-slave-6bbx"
server_key: "393a0bf2b9c36bdb3232b962fff54b1f"
server_key_iv: "c309eb33a78608f1215db7aed68f9580"
server_key_version: "encryptionkey@0"
I20260812 03:04:09.821376 21065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:04:09.832653 21065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:04:09.833194 21065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:04:09.833349 21065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:04:09.833839 21065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:04:09.834704 21065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:04:09.834753 21065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:09.834801 21065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:04:09.834831 21065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:09.842803 21065 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:32933
I20260812 03:04:09.842854 21309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:32933 every 8 connection(s)
I20260812 03:04:09.843634 21065 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:04:09.844792 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 21065
I20260812 03:04:09.844882 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance
I20260812 03:04:09.845142 5506 external_mini_cluster.cc:1428] Setting key 131021d893e941f118189348d5df6135
I20260812 03:04:09.851541 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:09.845259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:39695 (local address 127.5.96.190:38553)
0812 03:04:09.845403 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:09.845408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:09.846442 (+ 1034us) server_negotiation.cc:407] Connection header received
0812 03:04:09.847730 (+ 1288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:09.847734 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:09.847783 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:09.847847 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:09.848600 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:09.849093 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:09.850092 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:09.850289 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:09.850351 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:09.851188 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:04:09.851191 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:04:09.851193 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:04:09.851203 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:04:09.851223 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:04:09.851230 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:04:09.851290 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:09.851390 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 03:04:09.851437 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:04:09.852280 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:09.845654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:38553 (local address 127.5.96.131:39695)
0812 03:04:09.846317 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:09.846365 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:09.847533 (+ 1168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:09.847988 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:09.847995 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:09.848072 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:09.848501 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:09.848512 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:09.849174 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:09.849177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:09.850006 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:09.850015 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:09.850386 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:09.850389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:09.850512 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:09.850977 (+ 465us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:04:09.850978 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:09.851014 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:04:09.851069 (+ 55us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:04:09.851297 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:04:09.851301 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:04:09.851304 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:04:09.851385 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 03:04:09.851578 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":99,"threads_started":1}
I20260812 03:04:09.853214 21311 heartbeater.cc:347] Connected to a master server at 127.5.96.190:38553
I20260812 03:04:09.853402 21311 heartbeater.cc:464] Registering TS with master...
I20260812 03:04:09.853843 21311 heartbeater.cc:511] Master 127.5.96.190:38553 requested a full tablet report, sending...
I20260812 03:04:09.854698 20438 ts_manager.cc:194] Registered new tserver with Master: ccc4b6ca5e7c408fb63883702398690f (127.5.96.131:32933)
I20260812 03:04:09.855264 20438 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.96.131:39695
I20260812 03:04:09.858481 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 03:04:09.860965 20516 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:09.860258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33942 (local address 127.5.96.190:38553)
0812 03:04:09.860343 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:09.860347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:09.860519 (+ 172us) server_negotiation.cc:407] Connection header received
0812 03:04:09.860612 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:09.860614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:09.860645 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:09.860702 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:09.860880 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33942: BlockingRecv error: recv got EOF from 127.0.0.1:33942 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 03:04:09.861301 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 20562
I20260812 03:04:09.874733 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 20812
I20260812 03:04:09.891412 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 21065
I20260812 03:04:09.898782 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 20386
2026-08-12T03:04:09Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (847 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T03:04:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:04:09Z Disabled control of system clock
I20260812 03:04:09.933326 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:42313
--webserver_interface=127.5.96.190
--webserver_port=0
--builtin_ntp_servers=127.5.96.148:44397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:42313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 03:04:10.040344 21334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:04:10.040567 21334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:04:10.040601 21334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:04:10.044661 21334 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:04:10.044713 21334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:04:10.044727 21334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:04:10.044744 21334 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:04:10.044760 21334 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:04:10.049288 21334 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:44397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:42313
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:42313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:04:10.050160 21334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:04:10.051366 21334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:04:10.059127 21346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:10.059099 21341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:04:10.059201 21334 server_base.cc:1034] running on GCE node
W20260812 03:04:10.059098 21339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:04:10.059875 21334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:04:10.060824 21334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:04:10.061976 21334 hybrid_clock.cc:648] HybridClock initialized: now 1786503850061965 us; error 26 us; skew 500 ppm
I20260812 03:04:10.063720 21334 webserver.cc:460] Webserver started at http://127.5.96.190:42629/ using document root <none> and password file <none>
I20260812 03:04:10.064240 21334 fs_manager.cc:359] Metadata directory not provided
I20260812 03:04:10.064317 21334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:04:10.064510 21334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:04:10.066406 21334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "653195b12b85434d83689f5caa2a5141"
format_stamp: "Formatted at 2026-08-12 03:04:10 on dist-test-slave-6bbx"
server_key: "ed0c92c116c4bc35f2e06f74bd0b0a46"
server_key_iv: "97131d4e4038bc75f0599a3364fd6649"
server_key_version: "encryptionkey@0"
I20260812 03:04:10.066924 21334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "653195b12b85434d83689f5caa2a5141"
format_stamp: "Formatted at 2026-08-12 03:04:10 on dist-test-slave-6bbx"
server_key: "ed0c92c116c4bc35f2e06f74bd0b0a46"
server_key_iv: "97131d4e4038bc75f0599a3364fd6649"
server_key_version: "encryptionkey@0"
I20260812 03:04:10.070331 21334 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 03:04:10.072501 21355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:10.073658 21334 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:04:10.073776 21334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "653195b12b85434d83689f5caa2a5141"
format_stamp: "Formatted at 2026-08-12 03:04:10 on dist-test-slave-6bbx"
server_key: "ed0c92c116c4bc35f2e06f74bd0b0a46"
server_key_iv: "97131d4e4038bc75f0599a3364fd6649"
server_key_version: "encryptionkey@0"
I20260812 03:04:10.073872 21334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:04:10.078248 21334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:04:10.078828 21334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:04:10.078979 21334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:04:10.085278 21334 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:42313
I20260812 03:04:10.085299 21451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:42313 every 8 connection(s)
I20260812 03:04:10.086076 21334 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:04:10.087942 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 21334
I20260812 03:04:10.088047 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:04:10.088313 5506 external_mini_cluster.cc:1428] Setting key c726b8eb3cee961fd8ca455e9721206c
I20260812 03:04:10.089407 21453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:04:10.093520 21462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:10.089697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43758 (local address 127.5.96.190:42313)
0812 03:04:10.090309 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:10.090323 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:10.090356 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:04:10.090965 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:10.090981 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:10.091291 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:10.091580 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:10.092372 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:04:10.092376 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:04:10.092388 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:04:10.092397 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:04:10.092450 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:04:10.092465 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:04:10.092588 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:10.092679 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 03:04:10.092908 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":139,"threads_started":1}
I20260812 03:04:10.095264 21453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 653195b12b85434d83689f5caa2a5141: Bootstrap starting.
I20260812 03:04:10.097366 21453 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 653195b12b85434d83689f5caa2a5141: Neither blocks nor log segments found. Creating new log.
I20260812 03:04:10.098084 21453 log.cc:824] T 00000000000000000000000000000000 P 653195b12b85434d83689f5caa2a5141: Log is configured to *not* fsync() on all Append() calls
I20260812 03:04:10.099855 21453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 653195b12b85434d83689f5caa2a5141: No bootstrap required, opened a new log
I20260812 03:04:10.103223 21453 raft_consensus.cc:348] T 00000000000000000000000000000000 P 653195b12b85434d83689f5caa2a5141 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653195b12b85434d83689f5caa2a5141" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42313 } }
I20260812 03:04:10.103442 21453 raft_consensus.cc:374] T 00000000000000000000000000000000 P 653195b12b85434d83689f5caa2a5141 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:04:10.103502 21453 raft_consensus.cc:729] T 00000000000000000000000000000000 P 653195b12b85434d83689f5caa2a5141 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 653195b12b85434d83689f5caa2a5141, State: Initialized, Role: FOLLOWER
I20260812 03:04:10.103929 21453 consensus_queue.cc:260] T 00000000000000000000000000000000 P 653195b12b85434d83689f5caa2a5141 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653195b12b85434d83689f5caa2a5141" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42313 } }
I20260812 03:04:10.104059 21453 raft_consensus.cc:388] T 00000000000000000000000000000000 P 653195b12b85434d83689f5caa2a5141 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:04:10.104125 21453 raft_consensus.cc:482] T 00000000000000000000000000000000 P 653195b12b85434d83689f5caa2a5141 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:04:10.104220 21453 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 653195b12b85434d83689f5caa2a5141 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:04:10.105182 21453 raft_consensus.cc:504] T 00000000000000000000000000000000 P 653195b12b85434d83689f5caa2a5141 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653195b12b85434d83689f5caa2a5141" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42313 } }
I20260812 03:04:10.105486 21453 leader_election.cc:302] T 00000000000000000000000000000000 P 653195b12b85434d83689f5caa2a5141 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 653195b12b85434d83689f5caa2a5141; no voters:
I20260812 03:04:10.105762 21453 leader_election.cc:288] T 00000000000000000000000000000000 P 653195b12b85434d83689f5caa2a5141 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:04:10.105862 21466 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 653195b12b85434d83689f5caa2a5141 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:04:10.106056 21466 raft_consensus.cc:686] T 00000000000000000000000000000000 P 653195b12b85434d83689f5caa2a5141 [term 1 LEADER]: Becoming Leader. State: Replica: 653195b12b85434d83689f5caa2a5141, State: Running, Role: LEADER
I20260812 03:04:10.106407 21466 consensus_queue.cc:237] T 00000000000000000000000000000000 P 653195b12b85434d83689f5caa2a5141 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653195b12b85434d83689f5caa2a5141" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42313 } }
I20260812 03:04:10.106863 21453 sys_catalog.cc:564] T 00000000000000000000000000000000 P 653195b12b85434d83689f5caa2a5141 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:04:10.108166 21468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 653195b12b85434d83689f5caa2a5141 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 653195b12b85434d83689f5caa2a5141. Latest consensus state: current_term: 1 leader_uuid: "653195b12b85434d83689f5caa2a5141" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653195b12b85434d83689f5caa2a5141" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42313 } } }
I20260812 03:04:10.108197 21467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 653195b12b85434d83689f5caa2a5141 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "653195b12b85434d83689f5caa2a5141" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653195b12b85434d83689f5caa2a5141" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42313 } } }
I20260812 03:04:10.108289 21468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 653195b12b85434d83689f5caa2a5141 [sys.catalog]: This master's current role is: LEADER
I20260812 03:04:10.108292 21467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 653195b12b85434d83689f5caa2a5141 [sys.catalog]: This master's current role is: LEADER
I20260812 03:04:10.108659 21477 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:04:10.111480 21477 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:04:10.116576 21477 catalog_manager.cc:1347] Generated new cluster ID: 3aa8221730f643309502e7831d4da829
I20260812 03:04:10.116636 21477 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:04:10.156766 21477 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:04:10.157059 21477 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:04:10.168037 21477 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 653195b12b85434d83689f5caa2a5141: Generated new TSK 0
I20260812 03:04:10.168627 21477 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:04:10.177068 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:0
--local_ip_for_outbound_sockets=127.5.96.129
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:42313
--builtin_ntp_servers=127.5.96.148:44397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 03:04:10.307580 21504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:04:10.307821 21504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:04:10.307857 21504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:04:10.311710 21504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:04:10.311756 21504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:04:10.311806 21504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:04:10.316279 21504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:44397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:42313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:04:10.317168 21504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:04:10.318388 21504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:04:10.327451 21511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:10.327486 21510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:04:10.327479 21504 server_base.cc:1034] running on GCE node
W20260812 03:04:10.327558 21516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:04:10.328040 21504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:04:10.328655 21504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:04:10.329813 21504 hybrid_clock.cc:648] HybridClock initialized: now 1786503850329796 us; error 31 us; skew 500 ppm
I20260812 03:04:10.331597 21504 webserver.cc:460] Webserver started at http://127.5.96.129:45983/ using document root <none> and password file <none>
I20260812 03:04:10.332155 21504 fs_manager.cc:359] Metadata directory not provided
I20260812 03:04:10.332223 21504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:04:10.332432 21504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:04:10.334415 21504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/data/instance:
uuid: "110c48bb314e4ebe93a7689abaa6a4ae"
format_stamp: "Formatted at 2026-08-12 03:04:10 on dist-test-slave-6bbx"
server_key: "a71dd4956ddb53b12c47bff7ff9f0ee4"
server_key_iv: "63d4d940dc78a0e8abd89d0838d88f3f"
server_key_version: "encryptionkey@0"
I20260812 03:04:10.334929 21504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance:
uuid: "110c48bb314e4ebe93a7689abaa6a4ae"
format_stamp: "Formatted at 2026-08-12 03:04:10 on dist-test-slave-6bbx"
server_key: "a71dd4956ddb53b12c47bff7ff9f0ee4"
server_key_iv: "63d4d940dc78a0e8abd89d0838d88f3f"
server_key_version: "encryptionkey@0"
I20260812 03:04:10.338392 21504 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260812 03:04:10.340782 21526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:10.341873 21504 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:04:10.341977 21504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "110c48bb314e4ebe93a7689abaa6a4ae"
format_stamp: "Formatted at 2026-08-12 03:04:10 on dist-test-slave-6bbx"
server_key: "a71dd4956ddb53b12c47bff7ff9f0ee4"
server_key_iv: "63d4d940dc78a0e8abd89d0838d88f3f"
server_key_version: "encryptionkey@0"
I20260812 03:04:10.342065 21504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:04:10.346400 21504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:04:10.346937 21504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:04:10.347085 21504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:04:10.347584 21504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:04:10.348435 21504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:04:10.348482 21504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:10.348531 21504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:04:10.348560 21504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:10.356714 21504 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:37935
I20260812 03:04:10.356772 21727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:37935 every 8 connection(s)
I20260812 03:04:10.357532 21504 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:04:10.363233 21462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:10.359242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:58047 (local address 127.5.96.190:42313)
0812 03:04:10.359373 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:10.359377 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:10.360364 (+ 987us) server_negotiation.cc:407] Connection header received
0812 03:04:10.361643 (+ 1279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:10.361647 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:10.361694 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:10.361770 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:10.362784 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:04:10.362787 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:04:10.362790 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:04:10.362802 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:04:10.362823 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:04:10.362831 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:04:10.362908 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:10.363040 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 03:04:10.363084 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:04:10.364014 21733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:10.359569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:42313 (local address 127.5.96.129:58047)
0812 03:04:10.360222 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:10.360259 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:10.361428 (+ 1169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:10.361920 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:10.361927 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:10.362010 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:10.362518 (+ 508us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:04:10.362519 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:10.362557 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:04:10.362622 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:04:10.362902 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:04:10.362910 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:04:10.362916 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:04:10.363070 (+ 154us) client_negotiation.cc:240] Negotiation successful
0812 03:04:10.363298 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":112,"threads_started":1}
I20260812 03:04:10.364480 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 21504
I20260812 03:04:10.364588 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance
I20260812 03:04:10.364941 5506 external_mini_cluster.cc:1428] Setting key 8d37febf47f1799b066d95ddd5b524ce
I20260812 03:04:10.365228 21729 heartbeater.cc:347] Connected to a master server at 127.5.96.190:42313
I20260812 03:04:10.365439 21729 heartbeater.cc:464] Registering TS with master...
I20260812 03:04:10.365849 21729 heartbeater.cc:511] Master 127.5.96.190:42313 requested a full tablet report, sending...
I20260812 03:04:10.366766 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:0
--local_ip_for_outbound_sockets=127.5.96.130
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:42313
--builtin_ntp_servers=127.5.96.148:44397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 03:04:10.367280 21386 ts_manager.cc:194] Registered new tserver with Master: 110c48bb314e4ebe93a7689abaa6a4ae (127.5.96.129:37935)
W20260812 03:04:10.476383 21734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:04:10.476595 21734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:04:10.476630 21734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:04:10.480623 21734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:04:10.480669 21734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:04:10.480738 21734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:04:10.485301 21734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:44397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:42313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:04:10.486189 21734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:04:10.487430 21734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:04:10.496297 21743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:10.496248 21742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:04:10.496416 21734 server_base.cc:1034] running on GCE node
W20260812 03:04:10.496306 21745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:04:10.496876 21734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:04:10.497444 21734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:04:10.498603 21734 hybrid_clock.cc:648] HybridClock initialized: now 1786503850498581 us; error 34 us; skew 500 ppm
I20260812 03:04:10.500272 21734 webserver.cc:460] Webserver started at http://127.5.96.130:42377/ using document root <none> and password file <none>
I20260812 03:04:10.500782 21734 fs_manager.cc:359] Metadata directory not provided
I20260812 03:04:10.500846 21734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:04:10.501039 21734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:04:10.502923 21734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/data/instance:
uuid: "b728997b523b48d585dd69264bd82ce5"
format_stamp: "Formatted at 2026-08-12 03:04:10 on dist-test-slave-6bbx"
server_key: "bcaf62992ad6c5c22b1a4f34f87e5cc3"
server_key_iv: "243539d0989147bdc9ddda99e4108057"
server_key_version: "encryptionkey@0"
I20260812 03:04:10.503443 21734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance:
uuid: "b728997b523b48d585dd69264bd82ce5"
format_stamp: "Formatted at 2026-08-12 03:04:10 on dist-test-slave-6bbx"
server_key: "bcaf62992ad6c5c22b1a4f34f87e5cc3"
server_key_iv: "243539d0989147bdc9ddda99e4108057"
server_key_version: "encryptionkey@0"
I20260812 03:04:10.506857 21734 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:04:10.509262 21758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:10.510426 21734 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:04:10.510535 21734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "b728997b523b48d585dd69264bd82ce5"
format_stamp: "Formatted at 2026-08-12 03:04:10 on dist-test-slave-6bbx"
server_key: "bcaf62992ad6c5c22b1a4f34f87e5cc3"
server_key_iv: "243539d0989147bdc9ddda99e4108057"
server_key_version: "encryptionkey@0"
I20260812 03:04:10.510630 21734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:04:10.514961 21734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:04:10.515535 21734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:04:10.515718 21734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:04:10.516232 21734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:04:10.517083 21734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:04:10.517139 21734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:10.517187 21734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:04:10.517217 21734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:10.525545 21734 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:32957
I20260812 03:04:10.525566 21977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:32957 every 8 connection(s)
I20260812 03:04:10.526389 21734 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:04:10.531860 21462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:10.528060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:54949 (local address 127.5.96.190:42313)
0812 03:04:10.528202 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:10.528205 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:10.529147 (+ 942us) server_negotiation.cc:407] Connection header received
0812 03:04:10.530364 (+ 1217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:10.530366 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:10.530404 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:10.530464 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:10.531404 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:04:10.531407 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:04:10.531409 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:04:10.531421 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:04:10.531442 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:04:10.531450 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:04:10.531519 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:10.531603 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 03:04:10.531640 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 03:04:10.532596 21988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:10.528383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:42313 (local address 127.5.96.130:54949)
0812 03:04:10.529002 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:10.529041 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:10.530176 (+ 1135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:10.530587 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:10.530594 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:10.530677 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:10.531153 (+ 476us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:04:10.531154 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:10.531192 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:04:10.531261 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:04:10.531507 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:04:10.531511 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:04:10.531514 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:04:10.531629 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 03:04:10.531853 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":107,"threads_started":1}
I20260812 03:04:10.533145 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 21734
I20260812 03:04:10.533267 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance
I20260812 03:04:10.533586 5506 external_mini_cluster.cc:1428] Setting key 968548b300fcefe80130651ed25476e9
I20260812 03:04:10.533743 21983 heartbeater.cc:347] Connected to a master server at 127.5.96.190:42313
I20260812 03:04:10.533947 21983 heartbeater.cc:464] Registering TS with master...
I20260812 03:04:10.534358 21983 heartbeater.cc:511] Master 127.5.96.190:42313 requested a full tablet report, sending...
I20260812 03:04:10.535343 21386 ts_manager.cc:194] Registered new tserver with Master: b728997b523b48d585dd69264bd82ce5 (127.5.96.130:32957)
I20260812 03:04:10.535557 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:0
--local_ip_for_outbound_sockets=127.5.96.131
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:42313
--builtin_ntp_servers=127.5.96.148:44397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 03:04:10.645283 21989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:04:10.645489 21989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:04:10.645525 21989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:04:10.649385 21989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:04:10.649430 21989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:04:10.649480 21989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:04:10.653967 21989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:44397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:42313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:04:10.654804 21989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:04:10.656008 21989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:04:10.664826 22000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:10.664832 22002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:10.664906 21999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:04:10.665109 21989 server_base.cc:1034] running on GCE node
I20260812 03:04:10.665491 21989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:04:10.666052 21989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:04:10.667213 21989 hybrid_clock.cc:648] HybridClock initialized: now 1786503850667191 us; error 33 us; skew 500 ppm
I20260812 03:04:10.669067 21989 webserver.cc:460] Webserver started at http://127.5.96.131:33371/ using document root <none> and password file <none>
I20260812 03:04:10.669595 21989 fs_manager.cc:359] Metadata directory not provided
I20260812 03:04:10.669669 21989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:04:10.669858 21989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:04:10.671758 21989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/data/instance:
uuid: "9ed69dc6a6ff4bc4a20dc6c2b307e5a9"
format_stamp: "Formatted at 2026-08-12 03:04:10 on dist-test-slave-6bbx"
server_key: "3032c80ce4caf93cb94383f1d6cb0d53"
server_key_iv: "107d077ef45a40dc3a28535f275a46aa"
server_key_version: "encryptionkey@0"
I20260812 03:04:10.672281 21989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance:
uuid: "9ed69dc6a6ff4bc4a20dc6c2b307e5a9"
format_stamp: "Formatted at 2026-08-12 03:04:10 on dist-test-slave-6bbx"
server_key: "3032c80ce4caf93cb94383f1d6cb0d53"
server_key_iv: "107d077ef45a40dc3a28535f275a46aa"
server_key_version: "encryptionkey@0"
I20260812 03:04:10.675571 21989 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:04:10.677906 22013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:10.678987 21989 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:04:10.679090 21989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "9ed69dc6a6ff4bc4a20dc6c2b307e5a9"
format_stamp: "Formatted at 2026-08-12 03:04:10 on dist-test-slave-6bbx"
server_key: "3032c80ce4caf93cb94383f1d6cb0d53"
server_key_iv: "107d077ef45a40dc3a28535f275a46aa"
server_key_version: "encryptionkey@0"
I20260812 03:04:10.679181 21989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:04:10.683404 21989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:04:10.683976 21989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:04:10.684126 21989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:04:10.684612 21989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:04:10.685431 21989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:04:10.685477 21989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:10.685524 21989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:04:10.685554 21989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:10.693710 21989 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:33835
I20260812 03:04:10.693729 22226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:33835 every 8 connection(s)
I20260812 03:04:10.694470 21989 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:04:10.699985 21462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:10.696052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:33381 (local address 127.5.96.190:42313)
0812 03:04:10.696188 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:10.696192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:10.697123 (+ 931us) server_negotiation.cc:407] Connection header received
0812 03:04:10.698378 (+ 1255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:10.698382 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:10.698427 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:10.698509 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:10.699526 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:04:10.699529 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:04:10.699532 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:04:10.699546 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:04:10.699567 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:04:10.699574 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:04:10.699650 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:10.699805 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 03:04:10.699849 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:04:10.700748 22232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:10.696390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:42313 (local address 127.5.96.131:33381)
0812 03:04:10.696988 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:10.697025 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:10.698172 (+ 1147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:10.698674 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:10.698681 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:10.698763 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:10.699273 (+ 510us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:04:10.699274 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:10.699312 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:04:10.699376 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:04:10.699649 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:04:10.699658 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:04:10.699686 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 03:04:10.699834 (+ 148us) client_negotiation.cc:240] Negotiation successful
0812 03:04:10.700050 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":107,"threads_started":1}
I20260812 03:04:10.701725 22229 heartbeater.cc:347] Connected to a master server at 127.5.96.190:42313
I20260812 03:04:10.701926 22229 heartbeater.cc:464] Registering TS with master...
I20260812 03:04:10.702255 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 21989
I20260812 03:04:10.702294 22229 heartbeater.cc:511] Master 127.5.96.190:42313 requested a full tablet report, sending...
I20260812 03:04:10.702339 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance
I20260812 03:04:10.702589 5506 external_mini_cluster.cc:1428] Setting key 1a18e226cee0d3169369a9dbfce12779
I20260812 03:04:10.703223 21386 ts_manager.cc:194] Registered new tserver with Master: 9ed69dc6a6ff4bc4a20dc6c2b307e5a9 (127.5.96.131:33835)
I20260812 03:04:10.704617 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 03:04:10.707113 21462 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:04:10.706377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43770 (local address 127.5.96.190:42313)
0812 03:04:10.706475 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:10.706478 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:10.706648 (+ 170us) server_negotiation.cc:407] Connection header received
0812 03:04:10.706755 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:10.706758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:10.706785 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:10.706839 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:10.707006 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43770: BlockingRecv error: recv got EOF from 127.0.0.1:43770 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 03:04:10.707367 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 21504
I20260812 03:04:10.726266 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 21734
I20260812 03:04:10.737083 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 21989
I20260812 03:04:10.744622 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 21334
2026-08-12T03:04:10Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (842 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:04:10 dist-test-slave-6bbx krb5kdc[22249](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:04:10 dist-test-slave-6bbx krb5kdc[22249](info): set up 2 sockets
Aug 12 03:04:10 dist-test-slave-6bbx krb5kdc[22249](info): commencing operation
krb5kdc: starting...
W20260812 03:04:11.901980 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:04:11 dist-test-slave-6bbx krb5kdc[22249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:04:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:04:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:04:12.011619 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:44475
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:40911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:44475
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:04:12.119118 22280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:04:12.119365 22280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:04:12.119421 22280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:04:12.123320 22280 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:04:12.123375 22280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:04:12.123401 22280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:04:12.123421 22280 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:04:12.123447 22280 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:04:12.128520 22280 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:44475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:44475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:04:12.129467 22280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:04:12.130425 22280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:04:12.138499 22280 server_base.cc:1034] running on GCE node
W20260812 03:04:12.138445 22293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:12.138422 22294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:12.138429 22299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:04:12.139206 22280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:04:12.140249 22280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:04:12.141405 22280 hybrid_clock.cc:648] HybridClock initialized: now 1786503852141393 us; error 26 us; skew 500 ppm
Aug 12 03:04:12 dist-test-slave-6bbx krb5kdc[22249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503852, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:04:12.143956 22280 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:04:12.144753 22280 webserver.cc:460] Webserver started at http://127.5.96.190:35997/ using document root <none> and password file <none>
I20260812 03:04:12.145219 22280 fs_manager.cc:359] Metadata directory not provided
I20260812 03:04:12.145272 22280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:04:12.145439 22280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:04:12.147301 22280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "dd80f2b968cc4076a11230fd6eff7c35"
format_stamp: "Formatted at 2026-08-12 03:04:12 on dist-test-slave-6bbx"
server_key: "ca820fbdc1e4658ec99fb721749917bb"
server_key_iv: "52acf49a05f7230bcc725048454f370e"
server_key_version: "encryptionkey@0"
I20260812 03:04:12.147840 22280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "dd80f2b968cc4076a11230fd6eff7c35"
format_stamp: "Formatted at 2026-08-12 03:04:12 on dist-test-slave-6bbx"
server_key: "ca820fbdc1e4658ec99fb721749917bb"
server_key_iv: "52acf49a05f7230bcc725048454f370e"
server_key_version: "encryptionkey@0"
I20260812 03:04:12.151140 22280 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:04:12.153287 22314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:12.154381 22280 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260812 03:04:12.154486 22280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "dd80f2b968cc4076a11230fd6eff7c35"
format_stamp: "Formatted at 2026-08-12 03:04:12 on dist-test-slave-6bbx"
server_key: "ca820fbdc1e4658ec99fb721749917bb"
server_key_iv: "52acf49a05f7230bcc725048454f370e"
server_key_version: "encryptionkey@0"
I20260812 03:04:12.154577 22280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:04:12.173079 22280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:04:12.175434 22280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:04:12.175588 22280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:04:12.182026 22280 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:44475
I20260812 03:04:12.182315 22433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:44475 every 8 connection(s)
I20260812 03:04:12.182946 22280 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:04:12.185520 22439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:04:12.186520 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 22280
I20260812 03:04:12.186646 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:04:12.186967 5506 external_mini_cluster.cc:1428] Setting key e0a82597ebce4fa4e3b59d0b5eb33d91
I20260812 03:04:12.191633 22439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd80f2b968cc4076a11230fd6eff7c35: Bootstrap starting.
I20260812 03:04:12.194185 22439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd80f2b968cc4076a11230fd6eff7c35: Neither blocks nor log segments found. Creating new log.
I20260812 03:04:12.195057 22439 log.cc:824] T 00000000000000000000000000000000 P dd80f2b968cc4076a11230fd6eff7c35: Log is configured to *not* fsync() on all Append() calls
Aug 12 03:04:12 dist-test-slave-6bbx krb5kdc[22249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:04:12.196779 22439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd80f2b968cc4076a11230fd6eff7c35: No bootstrap required, opened a new log
I20260812 03:04:12.200090 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:12.188344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34420 (local address 127.5.96.190:44475)
0812 03:04:12.188730 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:12.188746 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:12.188779 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:04:12.189397 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:12.189412 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:12.189711 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:12.189973 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:12.190794 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.191750 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.192858 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.193307 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.193412 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:12.196036 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:04:12.196041 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:04:12.196053 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:04:12.196079 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:04:12.197560 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:04:12.197963 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:04:12.197967 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:04:12.197972 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:04:12.198076 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:04:12.198330 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:04:12.198333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:04:12.198335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:04:12.198681 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:04:12.198867 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:12.199120 (+ 253us) server_negotiation.cc:299] Negotiation successful
0812 03:04:12.199382 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":110,"threads_started":1}
I20260812 03:04:12.200075 22439 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd80f2b968cc4076a11230fd6eff7c35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd80f2b968cc4076a11230fd6eff7c35" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 44475 } }
I20260812 03:04:12.200350 22439 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd80f2b968cc4076a11230fd6eff7c35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:04:12.200414 22439 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd80f2b968cc4076a11230fd6eff7c35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd80f2b968cc4076a11230fd6eff7c35, State: Initialized, Role: FOLLOWER
I20260812 03:04:12.200846 22439 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd80f2b968cc4076a11230fd6eff7c35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd80f2b968cc4076a11230fd6eff7c35" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 44475 } }
I20260812 03:04:12.200997 22439 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd80f2b968cc4076a11230fd6eff7c35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:04:12.201063 22439 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd80f2b968cc4076a11230fd6eff7c35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:04:12.201160 22439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd80f2b968cc4076a11230fd6eff7c35 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:04:12.201932 22439 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd80f2b968cc4076a11230fd6eff7c35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd80f2b968cc4076a11230fd6eff7c35" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 44475 } }
I20260812 03:04:12.202291 22439 leader_election.cc:302] T 00000000000000000000000000000000 P dd80f2b968cc4076a11230fd6eff7c35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd80f2b968cc4076a11230fd6eff7c35; no voters:
I20260812 03:04:12.202550 22439 leader_election.cc:288] T 00000000000000000000000000000000 P dd80f2b968cc4076a11230fd6eff7c35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:04:12.202641 22450 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd80f2b968cc4076a11230fd6eff7c35 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:04:12.202843 22450 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd80f2b968cc4076a11230fd6eff7c35 [term 1 LEADER]: Becoming Leader. State: Replica: dd80f2b968cc4076a11230fd6eff7c35, State: Running, Role: LEADER
I20260812 03:04:12.203176 22450 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd80f2b968cc4076a11230fd6eff7c35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd80f2b968cc4076a11230fd6eff7c35" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 44475 } }
I20260812 03:04:12.203553 22439 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd80f2b968cc4076a11230fd6eff7c35 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:04:12.204733 22454 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd80f2b968cc4076a11230fd6eff7c35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd80f2b968cc4076a11230fd6eff7c35. Latest consensus state: current_term: 1 leader_uuid: "dd80f2b968cc4076a11230fd6eff7c35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd80f2b968cc4076a11230fd6eff7c35" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 44475 } } }
I20260812 03:04:12.204735 22453 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd80f2b968cc4076a11230fd6eff7c35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd80f2b968cc4076a11230fd6eff7c35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd80f2b968cc4076a11230fd6eff7c35" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 44475 } } }
I20260812 03:04:12.204910 22454 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd80f2b968cc4076a11230fd6eff7c35 [sys.catalog]: This master's current role is: LEADER
I20260812 03:04:12.204910 22453 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd80f2b968cc4076a11230fd6eff7c35 [sys.catalog]: This master's current role is: LEADER
I20260812 03:04:12.205329 22469 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:04:12.208001 22469 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:04:12.212759 22469 catalog_manager.cc:1347] Generated new cluster ID: 316e944317e94abcb7684a0f4d07bfce
I20260812 03:04:12.212807 22469 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:04:12.229758 22469 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:04:12.230903 22469 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:04:12.238170 22469 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd80f2b968cc4076a11230fd6eff7c35: Generated new TSK 0
I20260812 03:04:12.238837 22469 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.96.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
I20260812 03:04:12.268626 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:0
--local_ip_for_outbound_sockets=127.5.96.129
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:44475
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:40911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:04:12.376138 22484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:04:12.376360 22484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:04:12.376403 22484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:04:12.380303 22484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:04:12.380352 22484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:04:12.380429 22484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:04:12.385531 22484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:44475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:04:12.386440 22484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:04:12.387375 22484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:04:12.396287 22484 server_base.cc:1034] running on GCE node
W20260812 03:04:12.396224 22497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:12.396236 22498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:12.396202 22506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:04:12.396834 22484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:04:12.397403 22484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:04:12.398561 22484 hybrid_clock.cc:648] HybridClock initialized: now 1786503852398534 us; error 37 us; skew 500 ppm
Aug 12 03:04:12 dist-test-slave-6bbx krb5kdc[22249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503852, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:04:12.401098 22484 init.cc:377] Logged in from keytab as kudu/127.5.96.129@KRBTEST.COM (short username kudu)
I20260812 03:04:12.401870 22484 webserver.cc:460] Webserver started at http://127.5.96.129:33057/ using document root <none> and password file <none>
I20260812 03:04:12.402386 22484 fs_manager.cc:359] Metadata directory not provided
I20260812 03:04:12.402452 22484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:04:12.402681 22484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:04:12.404722 22484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/data/instance:
uuid: "1aae99d6b3dc48c78fcfafe7249ace6f"
format_stamp: "Formatted at 2026-08-12 03:04:12 on dist-test-slave-6bbx"
server_key: "4a305c0b27e44c474daacab6b4e70a7a"
server_key_iv: "9d15b4ae4623b88947637d5a6583c3a4"
server_key_version: "encryptionkey@0"
I20260812 03:04:12.405362 22484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance:
uuid: "1aae99d6b3dc48c78fcfafe7249ace6f"
format_stamp: "Formatted at 2026-08-12 03:04:12 on dist-test-slave-6bbx"
server_key: "4a305c0b27e44c474daacab6b4e70a7a"
server_key_iv: "9d15b4ae4623b88947637d5a6583c3a4"
server_key_version: "encryptionkey@0"
I20260812 03:04:12.408794 22484 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:04:12.410935 22518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:12.412082 22484 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:04:12.412199 22484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "1aae99d6b3dc48c78fcfafe7249ace6f"
format_stamp: "Formatted at 2026-08-12 03:04:12 on dist-test-slave-6bbx"
server_key: "4a305c0b27e44c474daacab6b4e70a7a"
server_key_iv: "9d15b4ae4623b88947637d5a6583c3a4"
server_key_version: "encryptionkey@0"
I20260812 03:04:12.412285 22484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:04:12.423044 22484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:04:12.425470 22484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:04:12.425622 22484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:04:12.426134 22484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:04:12.426939 22484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:04:12.427002 22484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:12.427048 22484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:04:12.427078 22484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:12.435240 22484 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:33123
I20260812 03:04:12.435273 22727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:33123 every 8 connection(s)
I20260812 03:04:12.436038 22484 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:04:12.443789 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 22484
I20260812 03:04:12.443892 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance
I20260812 03:04:12.444175 5506 external_mini_cluster.cc:1428] Setting key 601a76210dce666d6780e09c9ecd2050
Aug 12 03:04:12 dist-test-slave-6bbx krb5kdc[22249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503852, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:04:12.449379 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:12.437699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:55705 (local address 127.5.96.190:44475)
0812 03:04:12.437841 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:12.437846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:12.438646 (+ 800us) server_negotiation.cc:407] Connection header received
0812 03:04:12.439931 (+ 1285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:12.439934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:12.439982 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:12.440051 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:12.441113 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.441629 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.442693 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.442907 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.442974 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:12.445990 (+ 3016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:04:12.445993 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:04:12.445996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:04:12.446023 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:04:12.447384 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:04:12.447901 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:04:12.447904 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:04:12.447905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:04:12.447988 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:04:12.448312 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:04:12.448315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:04:12.448316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:04:12.448493 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:04:12.448580 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:12.449077 (+ 497us) server_negotiation.cc:299] Negotiation successful
0812 03:04:12.449222 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 03:04:12.450120 22734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:12.438049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:44475 (local address 127.5.96.129:55705)
0812 03:04:12.438499 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:12.438539 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:12.439736 (+ 1197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:12.440197 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:12.440203 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:12.440538 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:12.440990 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:12.441004 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.441732 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.441735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:12.442600 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:12.442609 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.443011 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.443014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:12.443142 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:12.444319 (+ 1177us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:12.444337 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:12.445823 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:04:12.447496 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:04:12.447501 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:04:12.447511 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:04:12.447805 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:04:12.448072 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:04:12.448075 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:04:12.448077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:04:12.448212 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:04:12.448593 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:04:12.448599 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:04:12.448842 (+ 243us) client_negotiation.cc:769] Sending connection context
0812 03:04:12.448997 (+ 155us) client_negotiation.cc:240] Negotiation successful
0812 03:04:12.449295 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":106,"threads_started":1}
I20260812 03:04:12.451381 22731 heartbeater.cc:347] Connected to a master server at 127.5.96.190:44475
I20260812 03:04:12.451593 22731 heartbeater.cc:464] Registering TS with master...
I20260812 03:04:12.452067 22731 heartbeater.cc:511] Master 127.5.96.190:44475 requested a full tablet report, sending...
I20260812 03:04:12.453466 22354 ts_manager.cc:194] Registered new tserver with Master: 1aae99d6b3dc48c78fcfafe7249ace6f (127.5.96.129:33123)
WARNING: no policy specified for kudu/127.5.96.130@KRBTEST.COM; defaulting to no policy
I20260812 03:04:12.454618 22354 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.129@KRBTEST.COM'} at 127.5.96.129:55705
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
I20260812 03:04:12.468250 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:0
--local_ip_for_outbound_sockets=127.5.96.130
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:44475
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:40911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:04:12.578510 22739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:04:12.578732 22739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:04:12.578769 22739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:04:12.582695 22739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:04:12.582795 22739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:04:12.582891 22739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:04:12.587839 22739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:44475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:04:12.588670 22739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:04:12.589543 22739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:04:12.598824 22749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:12.598839 22752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:04:12.598886 22739 server_base.cc:1034] running on GCE node
W20260812 03:04:12.598839 22747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:04:12.599514 22739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:04:12.600085 22739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:04:12.601235 22739 hybrid_clock.cc:648] HybridClock initialized: now 1786503852601220 us; error 25 us; skew 500 ppm
Aug 12 03:04:12 dist-test-slave-6bbx krb5kdc[22249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503852, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:04:12.603863 22739 init.cc:377] Logged in from keytab as kudu/127.5.96.130@KRBTEST.COM (short username kudu)
I20260812 03:04:12.604693 22739 webserver.cc:460] Webserver started at http://127.5.96.130:33303/ using document root <none> and password file <none>
I20260812 03:04:12.605175 22739 fs_manager.cc:359] Metadata directory not provided
I20260812 03:04:12.605275 22739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:04:12.605459 22739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:04:12.607218 22739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/data/instance:
uuid: "486008a7d200411ca686118423bce26e"
format_stamp: "Formatted at 2026-08-12 03:04:12 on dist-test-slave-6bbx"
server_key: "86f24830939c6a13a564eceefd5e2344"
server_key_iv: "e04d260265e19e6b9bab5165cfb2273e"
server_key_version: "encryptionkey@0"
I20260812 03:04:12.607733 22739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance:
uuid: "486008a7d200411ca686118423bce26e"
format_stamp: "Formatted at 2026-08-12 03:04:12 on dist-test-slave-6bbx"
server_key: "86f24830939c6a13a564eceefd5e2344"
server_key_iv: "e04d260265e19e6b9bab5165cfb2273e"
server_key_version: "encryptionkey@0"
I20260812 03:04:12.610937 22739 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:04:12.613083 22766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:12.614145 22739 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:04:12.614257 22739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "486008a7d200411ca686118423bce26e"
format_stamp: "Formatted at 2026-08-12 03:04:12 on dist-test-slave-6bbx"
server_key: "86f24830939c6a13a564eceefd5e2344"
server_key_iv: "e04d260265e19e6b9bab5165cfb2273e"
server_key_version: "encryptionkey@0"
I20260812 03:04:12.614343 22739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:04:12.625551 22739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:04:12.628280 22739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:04:12.628434 22739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:04:12.628921 22739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:04:12.629642 22739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:04:12.629686 22739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:12.629733 22739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:04:12.629763 22739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:12.637603 22739 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:35491
I20260812 03:04:12.637624 22989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:35491 every 8 connection(s)
I20260812 03:04:12.638350 22739 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:04:12.643225 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 22739
I20260812 03:04:12.643323 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance
I20260812 03:04:12.643587 5506 external_mini_cluster.cc:1428] Setting key acd8621ab9b640398f4ec6c4d774096e
Aug 12 03:04:12 dist-test-slave-6bbx krb5kdc[22249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503852, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:04:12.650705 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:12.639890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:33281 (local address 127.5.96.190:44475)
0812 03:04:12.640038 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:12.640043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:12.640817 (+ 774us) server_negotiation.cc:407] Connection header received
0812 03:04:12.641996 (+ 1179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:12.641998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:12.642044 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:12.642124 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:12.643119 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.643736 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.644827 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.645049 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.645135 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:12.647438 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:04:12.647441 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:04:12.647443 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:04:12.647466 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:04:12.648701 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:04:12.649214 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:04:12.649216 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:04:12.649218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:04:12.649287 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:04:12.649716 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:04:12.649718 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:04:12.649720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:04:12.649893 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:04:12.649964 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:12.650505 (+ 541us) server_negotiation.cc:299] Negotiation successful
0812 03:04:12.650594 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 03:04:12.651844 22997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:12.640260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:44475 (local address 127.5.96.130:33281)
0812 03:04:12.640676 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:12.640714 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:12.641826 (+ 1112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:12.642247 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:12.642253 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:12.642558 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:12.642992 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:12.643009 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.643854 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.643858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:12.644698 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:12.644708 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.645177 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.645181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:12.645331 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:12.645796 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:12.645812 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:12.647279 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:04:12.648812 (+ 1533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:04:12.648818 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:04:12.648831 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:04:12.649133 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:04:12.649374 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:04:12.649378 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:04:12.649380 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:04:12.649560 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:04:12.649989 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:04:12.649995 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:04:12.650283 (+ 288us) client_negotiation.cc:769] Sending connection context
0812 03:04:12.650487 (+ 204us) client_negotiation.cc:240] Negotiation successful
0812 03:04:12.650872 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":107,"threads_started":1}
I20260812 03:04:12.652897 22995 heartbeater.cc:347] Connected to a master server at 127.5.96.190:44475
I20260812 03:04:12.653102 22995 heartbeater.cc:464] Registering TS with master...
I20260812 03:04:12.653518 22995 heartbeater.cc:511] Master 127.5.96.190:44475 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.5.96.131@KRBTEST.COM; defaulting to no policy
I20260812 03:04:12.654455 22354 ts_manager.cc:194] Registered new tserver with Master: 486008a7d200411ca686118423bce26e (127.5.96.130:35491)
I20260812 03:04:12.655055 22354 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.130@KRBTEST.COM'} at 127.5.96.130:33281
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
I20260812 03:04:12.668356 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:0
--local_ip_for_outbound_sockets=127.5.96.131
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:44475
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:40911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:04:12.775164 23002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:04:12.775386 23002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:04:12.775442 23002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:04:12.779330 23002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:04:12.779381 23002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:04:12.779458 23002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:04:12.784474 23002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:44475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:04:12.785349 23002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:04:12.786244 23002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:04:12.795114 23018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:12.795153 23017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:12.795159 23022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:04:12.795476 23002 server_base.cc:1034] running on GCE node
I20260812 03:04:12.795854 23002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:04:12.796402 23002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:04:12.797544 23002 hybrid_clock.cc:648] HybridClock initialized: now 1786503852797510 us; error 46 us; skew 500 ppm
Aug 12 03:04:12 dist-test-slave-6bbx krb5kdc[22249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503852, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:04:12.803901 23002 init.cc:377] Logged in from keytab as kudu/127.5.96.131@KRBTEST.COM (short username kudu)
I20260812 03:04:12.804718 23002 webserver.cc:460] Webserver started at http://127.5.96.131:36669/ using document root <none> and password file <none>
I20260812 03:04:12.805243 23002 fs_manager.cc:359] Metadata directory not provided
I20260812 03:04:12.805315 23002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:04:12.805488 23002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:04:12.807376 23002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/data/instance:
uuid: "5b7b13b4322d4cd3a74c0019f69ebe4f"
format_stamp: "Formatted at 2026-08-12 03:04:12 on dist-test-slave-6bbx"
server_key: "400a9e4a382092569713b4f3acbade96"
server_key_iv: "d90f4175b113111c08c6b00e5def01a7"
server_key_version: "encryptionkey@0"
I20260812 03:04:12.807909 23002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance:
uuid: "5b7b13b4322d4cd3a74c0019f69ebe4f"
format_stamp: "Formatted at 2026-08-12 03:04:12 on dist-test-slave-6bbx"
server_key: "400a9e4a382092569713b4f3acbade96"
server_key_iv: "d90f4175b113111c08c6b00e5def01a7"
server_key_version: "encryptionkey@0"
I20260812 03:04:12.811259 23002 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260812 03:04:12.813448 23030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:12.814535 23002 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 03:04:12.814643 23002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "5b7b13b4322d4cd3a74c0019f69ebe4f"
format_stamp: "Formatted at 2026-08-12 03:04:12 on dist-test-slave-6bbx"
server_key: "400a9e4a382092569713b4f3acbade96"
server_key_iv: "d90f4175b113111c08c6b00e5def01a7"
server_key_version: "encryptionkey@0"
I20260812 03:04:12.814747 23002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:04:12.833894 23002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:04:12.836392 23002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:04:12.836547 23002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:04:12.837074 23002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:04:12.837826 23002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:04:12.837872 23002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:12.837911 23002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:04:12.837939 23002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:12.845956 23002 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:35429
I20260812 03:04:12.845985 23251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:35429 every 8 connection(s)
I20260812 03:04:12.846717 23002 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:04:12.853515 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 23002
I20260812 03:04:12.853611 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance
I20260812 03:04:12.853874 5506 external_mini_cluster.cc:1428] Setting key 6a20b460120ab87cbd399ed98690f4bc
Aug 12 03:04:12 dist-test-slave-6bbx krb5kdc[22249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503852, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:04:12.858776 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:12.848315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:42049 (local address 127.5.96.190:44475)
0812 03:04:12.848426 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:12.848430 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:12.849165 (+ 735us) server_negotiation.cc:407] Connection header received
0812 03:04:12.850424 (+ 1259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:12.850427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:12.850489 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:12.850560 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:12.851635 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.852204 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.853278 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.853494 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.853584 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:12.856049 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:04:12.856052 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:04:12.856054 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:04:12.856078 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:04:12.857130 (+ 1052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:04:12.857564 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:04:12.857566 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:04:12.857568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:04:12.857640 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:04:12.857901 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:04:12.857903 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:04:12.857904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:04:12.858060 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:04:12.858130 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:12.858549 (+ 419us) server_negotiation.cc:299] Negotiation successful
0812 03:04:12.858652 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 03:04:12.859436 23259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:12.848592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:44475 (local address 127.5.96.131:42049)
0812 03:04:12.849027 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:12.849066 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:12.850216 (+ 1150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:12.850735 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:12.850742 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:12.851059 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:12.851497 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:12.851509 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.852323 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.852326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:12.853157 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:12.853165 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.853628 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.853632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:12.853793 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:12.854291 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:12.854311 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:12.855912 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:04:12.857232 (+ 1320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:04:12.857236 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:04:12.857253 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:04:12.857488 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:04:12.857705 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:04:12.857707 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:04:12.857708 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:04:12.857838 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:04:12.858129 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:04:12.858134 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:04:12.858366 (+ 232us) client_negotiation.cc:769] Sending connection context
0812 03:04:12.858509 (+ 143us) client_negotiation.cc:240] Negotiation successful
0812 03:04:12.858728 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":123,"threads_started":1}
I20260812 03:04:12.860383 23254 heartbeater.cc:347] Connected to a master server at 127.5.96.190:44475
I20260812 03:04:12.860575 23254 heartbeater.cc:464] Registering TS with master...
I20260812 03:04:12.860983 23254 heartbeater.cc:511] Master 127.5.96.190:44475 requested a full tablet report, sending...
I20260812 03:04:12.861927 22354 ts_manager.cc:194] Registered new tserver with Master: 5b7b13b4322d4cd3a74c0019f69ebe4f (127.5.96.131:35429)
I20260812 03:04:12.862483 22354 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.131@KRBTEST.COM'} at 127.5.96.131:42049
I20260812 03:04:12.867329 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:04:12.875542 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:12.869030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34434 (local address 127.5.96.190:44475)
0812 03:04:12.869131 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:12.869134 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:12.869308 (+ 174us) server_negotiation.cc:407] Connection header received
0812 03:04:12.869415 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:12.869418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:12.869456 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:12.869517 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:12.870149 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.870624 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.871417 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.871620 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.871726 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:12.872875 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:04:12.872879 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:04:12.872881 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:04:12.872904 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:04:12.874079 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:04:12.874449 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:04:12.874453 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:04:12.874454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:04:12.874524 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:04:12.874762 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:04:12.874765 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:04:12.874767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:04:12.874961 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:04:12.875074 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:12.875302 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 03:04:12.875402 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 03:04:12.877951 22354 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34434:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:04:12.880852 22354 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:04:12.894430 23280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:12.889233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:35429 (local address 127.0.0.1:59328)
0812 03:04:12.889479 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:12.889491 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:12.889569 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:12.890873 (+ 1304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:12.890876 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:12.890895 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:04:12.890983 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:12.890988 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.892137 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.892140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:12.893200 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:12.893211 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.894019 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.894022 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:12.894111 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:12.894135 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:04:12.894213 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 03:04:12.894284 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":97,"threads_started":1}
I20260812 03:04:12.894596 23279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:12.888603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55312 (local address 127.5.96.130:35491)
0812 03:04:12.889445 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:12.889457 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:12.889479 (+ 22us) server_negotiation.cc:407] Connection header received
0812 03:04:12.889556 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:12.889562 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:12.889770 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:12.890253 (+ 483us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:04:12.890414 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.891481 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.892885 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.893663 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.893779 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:12.893915 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:12.894367 (+ 452us) server_negotiation.cc:299] Negotiation successful
0812 03:04:12.894463 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":741,"thread_start_us":138,"threads_started":1}
I20260812 03:04:12.894685 23281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:12.889525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51408 (local address 127.5.96.129:33123)
0812 03:04:12.889788 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:12.889797 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:12.889818 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:04:12.889895 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:12.889900 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:12.890086 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:12.890227 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:04:12.890442 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.891280 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.893363 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.894132 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.894243 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:12.894373 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:12.894460 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 03:04:12.894553 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":83,"threads_started":1}
I20260812 03:04:12.894297 23277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:12.888436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:35491 (local address 127.0.0.1:55312)
0812 03:04:12.888958 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:12.889006 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:12.889128 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:12.889946 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:12.889952 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:12.889976 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:04:12.890093 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:12.890101 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.891609 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.891613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:12.892760 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:12.892767 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.893789 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.893793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:12.893900 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:12.893942 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 03:04:12.894039 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 03:04:12.894152 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":67,"threads_started":1}
I20260812 03:04:12.894830 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:12.888587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:33123 (local address 127.0.0.1:51408)
0812 03:04:12.889051 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:12.889065 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:12.889148 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:12.890212 (+ 1064us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:12.890215 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:12.890233 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:04:12.890334 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:12.890341 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.891388 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.891391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:12.892738 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:12.892757 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.894259 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.894263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:12.894346 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:12.894369 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:04:12.894432 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 03:04:12.894486 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"mutex_wait_us":1,"thread_start_us":86,"threads_started":1}
I20260812 03:04:12.895151 23283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:12.890036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59328 (local address 127.5.96.131:35429)
0812 03:04:12.890467 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:12.890477 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:12.890497 (+ 20us) server_negotiation.cc:407] Connection header received
0812 03:04:12.890562 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:12.890567 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:12.890735 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:12.890880 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:04:12.891094 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.891985 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.893334 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:12.893909 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:12.894013 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:12.894139 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:12.894961 (+ 822us) server_negotiation.cc:299] Negotiation successful
0812 03:04:12.895051 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":136,"threads_started":1}
I20260812 03:04:12.897527 22611 tablet_service.cc:1467] Processing CreateTablet for tablet 2c43af451768474da39b50a0cd386842 (DEFAULT_TABLE table=test-table [id=839f6e2d0ca14175b166838ae4b69200]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:04:12.898448 22611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c43af451768474da39b50a0cd386842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:04:12.898157 23129 tablet_service.cc:1467] Processing CreateTablet for tablet 2c43af451768474da39b50a0cd386842 (DEFAULT_TABLE table=test-table [id=839f6e2d0ca14175b166838ae4b69200]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:04:12.898164 22861 tablet_service.cc:1467] Processing CreateTablet for tablet 2c43af451768474da39b50a0cd386842 (DEFAULT_TABLE table=test-table [id=839f6e2d0ca14175b166838ae4b69200]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:04:12.899171 22861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c43af451768474da39b50a0cd386842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:04:12.899180 23129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c43af451768474da39b50a0cd386842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:04:12.904109 23284 tablet_bootstrap.cc:492] T 2c43af451768474da39b50a0cd386842 P 1aae99d6b3dc48c78fcfafe7249ace6f: Bootstrap starting.
I20260812 03:04:12.905944 23285 tablet_bootstrap.cc:492] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e: Bootstrap starting.
I20260812 03:04:12.906665 23284 tablet_bootstrap.cc:654] T 2c43af451768474da39b50a0cd386842 P 1aae99d6b3dc48c78fcfafe7249ace6f: Neither blocks nor log segments found. Creating new log.
I20260812 03:04:12.906807 23286 tablet_bootstrap.cc:492] T 2c43af451768474da39b50a0cd386842 P 5b7b13b4322d4cd3a74c0019f69ebe4f: Bootstrap starting.
I20260812 03:04:12.907905 23284 log.cc:824] T 2c43af451768474da39b50a0cd386842 P 1aae99d6b3dc48c78fcfafe7249ace6f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:04:12.908532 23285 tablet_bootstrap.cc:654] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e: Neither blocks nor log segments found. Creating new log.
I20260812 03:04:12.909303 23286 tablet_bootstrap.cc:654] T 2c43af451768474da39b50a0cd386842 P 5b7b13b4322d4cd3a74c0019f69ebe4f: Neither blocks nor log segments found. Creating new log.
I20260812 03:04:12.909322 23285 log.cc:824] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:04:12.909938 23284 tablet_bootstrap.cc:492] T 2c43af451768474da39b50a0cd386842 P 1aae99d6b3dc48c78fcfafe7249ace6f: No bootstrap required, opened a new log
I20260812 03:04:12.910006 23286 log.cc:824] T 2c43af451768474da39b50a0cd386842 P 5b7b13b4322d4cd3a74c0019f69ebe4f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:04:12.910115 23284 ts_tablet_manager.cc:1397] T 2c43af451768474da39b50a0cd386842 P 1aae99d6b3dc48c78fcfafe7249ace6f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:04:12.911242 23285 tablet_bootstrap.cc:492] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e: No bootstrap required, opened a new log
I20260812 03:04:12.911432 23285 ts_tablet_manager.cc:1397] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:04:12.911597 23286 tablet_bootstrap.cc:492] T 2c43af451768474da39b50a0cd386842 P 5b7b13b4322d4cd3a74c0019f69ebe4f: No bootstrap required, opened a new log
I20260812 03:04:12.912071 23286 ts_tablet_manager.cc:1397] T 2c43af451768474da39b50a0cd386842 P 5b7b13b4322d4cd3a74c0019f69ebe4f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 03:04:12.913830 23284 raft_consensus.cc:348] T 2c43af451768474da39b50a0cd386842 P 1aae99d6b3dc48c78fcfafe7249ace6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486008a7d200411ca686118423bce26e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35491 } } peers { permanent_uuid: "1aae99d6b3dc48c78fcfafe7249ace6f" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33123 } } peers { permanent_uuid: "5b7b13b4322d4cd3a74c0019f69ebe4f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 35429 } }
I20260812 03:04:12.914088 23284 raft_consensus.cc:374] T 2c43af451768474da39b50a0cd386842 P 1aae99d6b3dc48c78fcfafe7249ace6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:04:12.914170 23284 raft_consensus.cc:729] T 2c43af451768474da39b50a0cd386842 P 1aae99d6b3dc48c78fcfafe7249ace6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aae99d6b3dc48c78fcfafe7249ace6f, State: Initialized, Role: FOLLOWER
I20260812 03:04:12.914608 23284 consensus_queue.cc:260] T 2c43af451768474da39b50a0cd386842 P 1aae99d6b3dc48c78fcfafe7249ace6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486008a7d200411ca686118423bce26e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35491 } } peers { permanent_uuid: "1aae99d6b3dc48c78fcfafe7249ace6f" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33123 } } peers { permanent_uuid: "5b7b13b4322d4cd3a74c0019f69ebe4f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 35429 } }
I20260812 03:04:12.914779 23286 raft_consensus.cc:348] T 2c43af451768474da39b50a0cd386842 P 5b7b13b4322d4cd3a74c0019f69ebe4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486008a7d200411ca686118423bce26e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35491 } } peers { permanent_uuid: "1aae99d6b3dc48c78fcfafe7249ace6f" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33123 } } peers { permanent_uuid: "5b7b13b4322d4cd3a74c0019f69ebe4f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 35429 } }
I20260812 03:04:12.915058 23286 raft_consensus.cc:374] T 2c43af451768474da39b50a0cd386842 P 5b7b13b4322d4cd3a74c0019f69ebe4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:04:12.915153 23286 raft_consensus.cc:729] T 2c43af451768474da39b50a0cd386842 P 5b7b13b4322d4cd3a74c0019f69ebe4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b7b13b4322d4cd3a74c0019f69ebe4f, State: Initialized, Role: FOLLOWER
I20260812 03:04:12.915033 23285 raft_consensus.cc:348] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486008a7d200411ca686118423bce26e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35491 } } peers { permanent_uuid: "1aae99d6b3dc48c78fcfafe7249ace6f" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33123 } } peers { permanent_uuid: "5b7b13b4322d4cd3a74c0019f69ebe4f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 35429 } }
I20260812 03:04:12.915230 22731 heartbeater.cc:503] Master 127.5.96.190:44475 was elected leader, sending a full tablet report...
I20260812 03:04:12.915450 23285 raft_consensus.cc:374] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:04:12.915534 23285 raft_consensus.cc:729] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 486008a7d200411ca686118423bce26e, State: Initialized, Role: FOLLOWER
I20260812 03:04:12.915606 23286 consensus_queue.cc:260] T 2c43af451768474da39b50a0cd386842 P 5b7b13b4322d4cd3a74c0019f69ebe4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486008a7d200411ca686118423bce26e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35491 } } peers { permanent_uuid: "1aae99d6b3dc48c78fcfafe7249ace6f" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33123 } } peers { permanent_uuid: "5b7b13b4322d4cd3a74c0019f69ebe4f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 35429 } }
I20260812 03:04:12.916009 23285 consensus_queue.cc:260] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486008a7d200411ca686118423bce26e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35491 } } peers { permanent_uuid: "1aae99d6b3dc48c78fcfafe7249ace6f" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33123 } } peers { permanent_uuid: "5b7b13b4322d4cd3a74c0019f69ebe4f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 35429 } }
I20260812 03:04:12.916242 23284 ts_tablet_manager.cc:1428] T 2c43af451768474da39b50a0cd386842 P 1aae99d6b3dc48c78fcfafe7249ace6f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 03:04:12.916556 23286 ts_tablet_manager.cc:1428] T 2c43af451768474da39b50a0cd386842 P 5b7b13b4322d4cd3a74c0019f69ebe4f: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 03:04:12.916853 22995 heartbeater.cc:503] Master 127.5.96.190:44475 was elected leader, sending a full tablet report...
I20260812 03:04:12.916901 23254 heartbeater.cc:503] Master 127.5.96.190:44475 was elected leader, sending a full tablet report...
I20260812 03:04:12.917146 23285 ts_tablet_manager.cc:1428] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
W20260812 03:04:12.937036 22733 tablet.cc:2369] T 2c43af451768474da39b50a0cd386842 P 1aae99d6b3dc48c78fcfafe7249ace6f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:04:13.065053 23296 raft_consensus.cc:482] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:04:13.065264 23296 raft_consensus.cc:504] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486008a7d200411ca686118423bce26e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35491 } } peers { permanent_uuid: "1aae99d6b3dc48c78fcfafe7249ace6f" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33123 } } peers { permanent_uuid: "5b7b13b4322d4cd3a74c0019f69ebe4f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 35429 } }
I20260812 03:04:13.066386 23296 leader_election.cc:288] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aae99d6b3dc48c78fcfafe7249ace6f (127.5.96.129:33123), 5b7b13b4322d4cd3a74c0019f69ebe4f (127.5.96.131:35429)
I20260812 03:04:13.070580 22997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:13.066618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:35429 (local address 127.5.96.130:50181)
0812 03:04:13.066728 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:13.066741 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:13.066849 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:13.067157 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:13.067184 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:13.067208 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:04:13.067334 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:13.067341 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:13.068277 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:13.068281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:13.069451 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:13.069463 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:13.070203 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:13.070206 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:13.070289 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:13.070311 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:04:13.070371 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 03:04:13.070441 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 03:04:13.070603 23283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:13.066758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:50181 (local address 127.5.96.131:35429)
0812 03:04:13.066897 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:13.066901 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:13.066915 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:04:13.066967 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:13.066971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:13.067020 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:13.067094 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:04:13.067460 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:13.068167 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:13.069568 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:13.070086 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:13.070198 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:13.070249 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:13.070406 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 03:04:13.070477 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:04:13.070925 23281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:13.066727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:52559 (local address 127.5.96.129:33123)
0812 03:04:13.066854 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:13.066857 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:13.067379 (+ 522us) server_negotiation.cc:407] Connection header received
0812 03:04:13.067484 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:13.067487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:13.067538 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:13.067617 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:04:13.067946 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:13.068529 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:13.069819 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:13.070354 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:13.070463 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:13.070506 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:13.070750 (+ 244us) server_negotiation.cc:299] Negotiation successful
0812 03:04:13.070831 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:04:13.071280 23300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:13.066618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:33123 (local address 127.5.96.130:52559)
0812 03:04:13.067272 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:13.067286 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:13.067392 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:13.067644 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:13.067647 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:13.067723 (+ 76us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:04:13.067832 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:13.067840 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:13.068651 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:13.068655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:13.069711 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:13.069720 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:13.070475 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:13.070479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:13.070569 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:13.070610 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 03:04:13.071072 (+ 462us) client_negotiation.cc:240] Negotiation successful
0812 03:04:13.071149 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"mutex_wait_us":292,"thread_start_us":115,"threads_started":1}
I20260812 03:04:13.071297 23159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c43af451768474da39b50a0cd386842" candidate_uuid: "486008a7d200411ca686118423bce26e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b7b13b4322d4cd3a74c0019f69ebe4f" is_pre_election: true
I20260812 03:04:13.071602 23159 raft_consensus.cc:2454] T 2c43af451768474da39b50a0cd386842 P 5b7b13b4322d4cd3a74c0019f69ebe4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 486008a7d200411ca686118423bce26e in term 0.
I20260812 03:04:13.071929 22637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c43af451768474da39b50a0cd386842" candidate_uuid: "486008a7d200411ca686118423bce26e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aae99d6b3dc48c78fcfafe7249ace6f" is_pre_election: true
I20260812 03:04:13.072147 22772 leader_election.cc:302] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 486008a7d200411ca686118423bce26e, 5b7b13b4322d4cd3a74c0019f69ebe4f; no voters:
I20260812 03:04:13.072218 22637 raft_consensus.cc:2454] T 2c43af451768474da39b50a0cd386842 P 1aae99d6b3dc48c78fcfafe7249ace6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 486008a7d200411ca686118423bce26e in term 0.
I20260812 03:04:13.072422 23296 raft_consensus.cc:2781] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:04:13.072525 23296 raft_consensus.cc:482] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:04:13.072575 23296 raft_consensus.cc:3037] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:04:13.073589 23296 raft_consensus.cc:504] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486008a7d200411ca686118423bce26e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35491 } } peers { permanent_uuid: "1aae99d6b3dc48c78fcfafe7249ace6f" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33123 } } peers { permanent_uuid: "5b7b13b4322d4cd3a74c0019f69ebe4f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 35429 } }
I20260812 03:04:13.073945 23296 leader_election.cc:288] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e [CANDIDATE]: Term 1 election: Requested vote from peers 1aae99d6b3dc48c78fcfafe7249ace6f (127.5.96.129:33123), 5b7b13b4322d4cd3a74c0019f69ebe4f (127.5.96.131:35429)
I20260812 03:04:13.074308 22637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c43af451768474da39b50a0cd386842" candidate_uuid: "486008a7d200411ca686118423bce26e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aae99d6b3dc48c78fcfafe7249ace6f"
I20260812 03:04:13.074295 23159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c43af451768474da39b50a0cd386842" candidate_uuid: "486008a7d200411ca686118423bce26e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b7b13b4322d4cd3a74c0019f69ebe4f"
I20260812 03:04:13.074410 22637 raft_consensus.cc:3037] T 2c43af451768474da39b50a0cd386842 P 1aae99d6b3dc48c78fcfafe7249ace6f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:04:13.074411 23159 raft_consensus.cc:3037] T 2c43af451768474da39b50a0cd386842 P 5b7b13b4322d4cd3a74c0019f69ebe4f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:04:13.075609 23159 raft_consensus.cc:2454] T 2c43af451768474da39b50a0cd386842 P 5b7b13b4322d4cd3a74c0019f69ebe4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 486008a7d200411ca686118423bce26e in term 1.
I20260812 03:04:13.075609 22637 raft_consensus.cc:2454] T 2c43af451768474da39b50a0cd386842 P 1aae99d6b3dc48c78fcfafe7249ace6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 486008a7d200411ca686118423bce26e in term 1.
I20260812 03:04:13.075920 22772 leader_election.cc:302] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 486008a7d200411ca686118423bce26e, 5b7b13b4322d4cd3a74c0019f69ebe4f; no voters:
I20260812 03:04:13.076107 23296 raft_consensus.cc:2781] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:04:13.076321 23296 raft_consensus.cc:686] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e [term 1 LEADER]: Becoming Leader. State: Replica: 486008a7d200411ca686118423bce26e, State: Running, Role: LEADER
I20260812 03:04:13.076663 23296 consensus_queue.cc:237] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486008a7d200411ca686118423bce26e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35491 } } peers { permanent_uuid: "1aae99d6b3dc48c78fcfafe7249ace6f" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33123 } } peers { permanent_uuid: "5b7b13b4322d4cd3a74c0019f69ebe4f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 35429 } }
I20260812 03:04:13.080143 22352 catalog_manager.cc:5495] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e reported cstate change: term changed from 0 to 1, leader changed from <none> to 486008a7d200411ca686118423bce26e (127.5.96.130). New cstate: current_term: 1 leader_uuid: "486008a7d200411ca686118423bce26e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486008a7d200411ca686118423bce26e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1aae99d6b3dc48c78fcfafe7249ace6f" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b7b13b4322d4cd3a74c0019f69ebe4f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 35429 } health_report { overall_health: UNKNOWN } } }
W20260812 03:04:13.097553 23255 tablet.cc:2369] T 2c43af451768474da39b50a0cd386842 P 5b7b13b4322d4cd3a74c0019f69ebe4f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:04:13.132483 23279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:13.129363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55326 (local address 127.5.96.130:35491)
0812 03:04:13.129489 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:13.129493 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:13.129506 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:04:13.129597 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:13.129599 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:13.129651 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:13.129734 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:04:13.129930 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:13.130417 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:13.131196 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:13.131400 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:13.131463 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:13.131717 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:04:13.132140 (+ 423us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:04:13.132231 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:13.132295 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 03:04:13.132343 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 03:04:13.138001 22637 raft_consensus.cc:1261] T 2c43af451768474da39b50a0cd386842 P 1aae99d6b3dc48c78fcfafe7249ace6f [term 1 FOLLOWER]: Refusing update from remote peer 486008a7d200411ca686118423bce26e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:04:13.138000 23159 raft_consensus.cc:1261] T 2c43af451768474da39b50a0cd386842 P 5b7b13b4322d4cd3a74c0019f69ebe4f [term 1 FOLLOWER]: Refusing update from remote peer 486008a7d200411ca686118423bce26e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:04:13.138715 23302 consensus_queue.cc:1035] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b7b13b4322d4cd3a74c0019f69ebe4f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 35429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:04:13.138900 23296 consensus_queue.cc:1035] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aae99d6b3dc48c78fcfafe7249ace6f" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 33123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 03:04:13.139537 22996 tablet.cc:2369] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:04:13.153896 22346 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:04:13.154045 22346 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:04:13.156272 22346 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dd80f2b968cc4076a11230fd6eff7c35: Sending DeleteTablet for 3 replicas of tablet 2c43af451768474da39b50a0cd386842
I20260812 03:04:13.156991 22611 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c43af451768474da39b50a0cd386842 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:04:13 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:51408
I20260812 03:04:13.157204 23129 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c43af451768474da39b50a0cd386842 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:04:13 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:59328
I20260812 03:04:13.157473 22861 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c43af451768474da39b50a0cd386842 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:04:13 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:55312
I20260812 03:04:13.157716 23339 tablet_replica.cc:330] T 2c43af451768474da39b50a0cd386842 P 5b7b13b4322d4cd3a74c0019f69ebe4f: stopping tablet replica
I20260812 03:04:13.158092 23339 raft_consensus.cc:2229] T 2c43af451768474da39b50a0cd386842 P 5b7b13b4322d4cd3a74c0019f69ebe4f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:04:13.158411 23339 raft_consensus.cc:2258] T 2c43af451768474da39b50a0cd386842 P 5b7b13b4322d4cd3a74c0019f69ebe4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:04:13.159447 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 22484
I20260812 03:04:13.159883 23339 ts_tablet_manager.cc:1905] T 2c43af451768474da39b50a0cd386842 P 5b7b13b4322d4cd3a74c0019f69ebe4f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:04:13.162818 23339 ts_tablet_manager.cc:1918] T 2c43af451768474da39b50a0cd386842 P 5b7b13b4322d4cd3a74c0019f69ebe4f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:04:13.162914 23339 log.cc:1196] T 2c43af451768474da39b50a0cd386842 P 5b7b13b4322d4cd3a74c0019f69ebe4f: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786503790637001-5506-0/minicluster-data/ts-2/wal/wals/2c43af451768474da39b50a0cd386842
I20260812 03:04:13.163218 23339 ts_tablet_manager.cc:1939] T 2c43af451768474da39b50a0cd386842 P 5b7b13b4322d4cd3a74c0019f69ebe4f: Deleting consensus metadata
I20260812 03:04:13.168301 22325 catalog_manager.cc:4841] TS 5b7b13b4322d4cd3a74c0019f69ebe4f (127.5.96.131:35429): tablet 2c43af451768474da39b50a0cd386842 (table test-table [id=839f6e2d0ca14175b166838ae4b69200]) successfully deleted
I20260812 03:04:13.174713 23341 tablet_replica.cc:330] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e: stopping tablet replica
W20260812 03:04:13.174742 22327 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.96.129:33123 (error 108)
I20260812 03:04:13.174793 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 22739
I20260812 03:04:13.175034 23341 raft_consensus.cc:2229] T 2c43af451768474da39b50a0cd386842 P 486008a7d200411ca686118423bce26e [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:04:13.180161 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:13.179822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:33123 (local address 127.0.0.1:51420)
0812 03:04:13.180002 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:13.180076 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.129:33123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260812 03:04:13.180328 22327 catalog_manager.cc:4584] TS 1aae99d6b3dc48c78fcfafe7249ace6f (127.5.96.129:33123): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2c43af451768474da39b50a0cd386842: Network error: Client connection negotiation failed: client connection to 127.5.96.129:33123: connect: Connection refused (error 111)
I20260812 03:04:13.186079 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 23002
I20260812 03:04:13.186178 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:13.185955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:35491 (local address 127.0.0.1:55330)
0812 03:04:13.186062 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:13.186112 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.130:35491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 03:04:13.213960 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 22280
2026-08-12T03:04:13Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2471 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:04:13 dist-test-slave-6bbx krb5kdc[23354](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:04:13 dist-test-slave-6bbx krb5kdc[23354](info): set up 2 sockets
Aug 12 03:04:13 dist-test-slave-6bbx krb5kdc[23354](info): commencing operation
krb5kdc: starting...
W20260812 03:04:14.372431 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:04:14 dist-test-slave-6bbx krb5kdc[23354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:04:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:04:14Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.96.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.96.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.96.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/rangeradmin_127.5.96.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.96.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/rangeradmin_127.5.96.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.96.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.96.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.96.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/rangerlookup_127.5.96.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.96.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/rangerlookup_127.5.96.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.96.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.96.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.96.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/HTTP_127.5.96.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.96.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/HTTP_127.5.96.148@KRBTEST.COM.keytab.
I20260812 03:04:14.521395 5506 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task6aqATZ/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/postgres -l logfile start
2026-08-12 03:04:16.590 UTC [23425] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 03:04:16.591 UTC [23425] LOG: listening on IPv4 address "127.5.96.148", port 40857
2026-08-12 03:04:16.593 UTC [23425] LOG: listening on Unix socket "/tmp/.s.PGSQL.40857"
2026-08-12 03:04:16.612 UTC [23432] LOG: database system was shut down at 2026-08-12 03:04:15 UTC
2026-08-12 03:04:16.615 UTC [23425] LOG: database system is ready to accept connections
I20260812 03:04:17.690220 5506 mini_postgres.cc:96] Postgres bound to 40857
2026-08-12 03:04:17.698 UTC [23448] FATAL: database "slave" does not exist
127.5.96.148:40857 - accepting connections
I20260812 03:04:17.699256 5506 mini_ranger.cc:161] Starting Ranger...
I20260812 03:04:17.714238 5506 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 03:04:18.054881 5506 mini_ranger.cc:87] Created ranger Postgres database
I20260812 03:04:18.054960 5506 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ranger-admin
2026-08-12 03:04:18,376 [I] DB FLAVOR :POSTGRES
2026-08-12 03:04:18,376 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 03:04:18,376 [I] Checking connection..
2026-08-12 03:04:18,376 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:40857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 03:04:18,612 [I] Checking connection passed.
2026-08-12 03:04:18,612 [I] --------- Verifying version history table ---------
2026-08-12 03:04:18,612 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:40857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:04:18,866 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 03:04:18,867 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:40857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:04:19,114 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 03:04:19,115 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 03:04:19,115 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:40857/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 03:04:19.359 UTC [23563] WARNING: there is no transaction in progress
2026-08-12 03:04:19,367 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 03:04:19,367 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:40857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:04:19,623 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 03:04:19,624 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 03:04:19,624 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:40857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 03:04:19,852 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:40857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 03:04:20,102 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:40857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6bbx.c.gcp-upstream.internal','N') ;"
2026-08-12 03:04:20,337 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:40857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 03:04:20,564 [I] Table x_portal_user does not exist in database ranger
2026-08-12 03:04:20,564 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:40857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 03:04:20,807 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 03:04:20,808 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:40857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 03:04:21,045 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:40857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 03:04:21,293 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 03:04:21,293 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:40857/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 03:04:22.149 UTC [23798] WARNING: there is no transaction in progress
2026-08-12 03:04:22.163 UTC [23798] WARNING: there is no transaction in progress
2026-08-12 03:04:22.178 UTC [23798] WARNING: there is no transaction in progress
2026-08-12 03:04:22.192 UTC [23798] WARNING: there is no transaction in progress
2026-08-12 03:04:22.209 UTC [23798] WARNING: there is no transaction in progress
2026-08-12 03:04:22.227 UTC [23798] WARNING: there is no transaction in progress
2026-08-12 03:04:22.285 UTC [23798] WARNING: there is no transaction in progress
2026-08-12 03:04:22.291 UTC [23798] WARNING: there is no transaction in progress
2026-08-12 03:04:22.304 UTC [23798] WARNING: there is no transaction in progress
2026-08-12 03:04:22.313 UTC [23798] WARNING: there is no transaction in progress
2026-08-12 03:04:22.322 UTC [23798] WARNING: there is no transaction in progress
2026-08-12 03:04:22.335 UTC [23798] WARNING: there is no transaction in progress
2026-08-12 03:04:22.343 UTC [23798] WARNING: there is no transaction in progress
2026-08-12 03:04:22.349 UTC [23798] WARNING: there is no transaction in progress
2026-08-12 03:04:22.355 UTC [23798] WARNING: there is no transaction in progress
2026-08-12 03:04:22.360 UTC [23798] WARNING: there is no transaction in progress
2026-08-12 03:04:22,856 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 03:04:22,857 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:40857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 03:04:23,102 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 03:04:23,102 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:40857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 03:04:23,336 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 03:04:23,337 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:40857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 03:04:23,583 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 03:04:23,583 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:40857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 03:04:23,841 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:40857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 03:04:24,081 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:40857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6bbx.c.gcp-upstream.internal';"
2026-08-12 03:04:24,314 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 03:04:24,315 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:40857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 03:04:24,554 [I] DB_PATCHES have already been applied
I20260812 03:04:24.560168 5506 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ranger-admin:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task6aqATZ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 03:04:24.560231 5506 mini_ranger.cc:193] Using host: 127.5.96.148
I20260812 03:04:24.564539 5506 mini_ranger.cc:227] Ranger admin URL: http://127.5.96.148:34659
Aug 12, 2026 3:04:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 3:04:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 3:04:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 3:04:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 3:04:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 3:04:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 3:04:25 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 03:04:25.817346 5506 mini_ranger.cc:160] Time spent starting Ranger: real 8.118s user 0.006s sys 0.005s
Aug 12, 2026 3:04:44 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 3:04:44 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 12, 2026 3:04:44 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 3:04:44 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 3:04:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 3:04:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 3:04:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 3:04:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 3:04:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 3:04:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 3:04:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 3:04:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 3:04:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 3:04:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 3:04:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 3:04:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 3:04:44 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 3:04:45 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 03:04:45.575325 5506 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.96.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.96.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.96.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/rangerkms_127.5.96.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.96.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/rangerkms_127.5.96.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.96.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.96.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.96.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/HTTP_127.5.96.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.96.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/HTTP_127.5.96.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 03:04:45 dist-test-slave-6bbx krb5kdc[23354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503885, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 03:04:45.653201 5506 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 03:04:45.662220 5506 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 03:04:46.276333 5506 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 03:04:46.276420 5506 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ranger-kms
I20260812 03:04:46.276511 5506 mini_ranger_kms.cc:227] Using postgres at 127.5.96.148:40857
2026-08-12 03:04:46,666 [I] DB FLAVOR :POSTGRES
2026-08-12 03:04:46,666 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 03:04:46,667 [I] Checking connection
2026-08-12 03:04:46,667 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:40857/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 03:04:46,898 [I] connection success
2026-08-12 03:04:46,898 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 03:04:46,898 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 03:04:46,898 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:40857/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 03:04:47,154 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 03:04:47,154 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 03:04:47,154 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 03:04:47,154 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:40857/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 03:04:47,405 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 03:04:47.816114 5506 mini_ranger_kms.cc:310] Created kms service
I20260812 03:04:47.982956 5506 mini_ranger_kms.cc:326] Created kudu user
I20260812 03:04:48.020290 5506 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 03:04:48 dist-test-slave-6bbx krb5kdc[23354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503885, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.96.148@KRBTEST.COM
I20260812 03:04:48.362954 5506 mini_ranger_kms.cc:384] Added ranger policy
I20260812 03:04:48.363132 5506 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ranger-kms:/tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 03:04:48.363171 5506 mini_ranger_kms.cc:240] Using host: 127.5.96.148
I20260812 03:04:48.367820 5506 mini_ranger_kms.cc:276] Ranger KMS PID: 24136
I20260812 03:04:48.367956 5506 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.96.148:55607
Aug 12, 2026 3:04:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 3:04:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 3:04:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 3:04:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 3:04:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 3:04:49 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 03:04:50 dist-test-slave-6bbx krb5kdc[23354](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786503890, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.96.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:04:50.919539 5506 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.266s user 0.018s sys 0.265s
I20260812 03:04:50.919701 5506 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 03:04:50.919818 5506 mini_ranger_kms.cc:401] 127.5.96.148:55607/kms/v1/keys
Aug 12 03:04:51 dist-test-slave-6bbx krb5kdc[23354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503890, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.96.148@KRBTEST.COM for HTTP/127.5.96.148@KRBTEST.COM
Aug 12 03:04:51 dist-test-slave-6bbx krb5kdc[23354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503890, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.96.148@KRBTEST.COM for HTTP/127.5.96.148@KRBTEST.COM
Aug 12 03:04:53 dist-test-slave-6bbx krb5kdc[23354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:04:53.654507 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:40143
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:40143
--ranger_config_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.96.148:55607/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:04:53.766624 24236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:04:53.766862 24236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:04:53.766897 24236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:04:53.770753 24236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:04:53.770802 24236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:04:53.770819 24236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:04:53.770833 24236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:04:53.770848 24236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:04:53.775998 24236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.96.148:55607/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:40143
--ranger_config_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:40143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:04:53.776918 24236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:04:53.777904 24236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:04:53.786157 24247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:53.786170 24244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:53.786170 24245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:04:53.786485 24236 server_base.cc:1034] running on GCE node
I20260812 03:04:53.787137 24236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:04:53.788508 24236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:04:53.789692 24236 hybrid_clock.cc:648] HybridClock initialized: now 1786503893789670 us; error 29 us; skew 500 ppm
Aug 12 03:04:53 dist-test-slave-6bbx krb5kdc[23354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503893, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:04:53.796967 24236 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:04:53.798012 24236 webserver.cc:460] Webserver started at http://127.5.96.190:36765/ using document root <none> and password file <none>
I20260812 03:04:53.798648 24236 fs_manager.cc:359] Metadata directory not provided
I20260812 03:04:53.798720 24236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:04:53.798943 24236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:04:53 dist-test-slave-6bbx krb5kdc[23354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503893, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for HTTP/127.5.96.148@KRBTEST.COM
I20260812 03:04:53.845235 24236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "a5ee05a4153e497cbbc3c9619dadf760"
format_stamp: "Formatted at 2026-08-12 03:04:53 on dist-test-slave-6bbx"
server_key: "6db7307af6ab0185d92ebafed1015168"
server_key_iv: "b03b9c4b22e170a7ba0613c9c80f4d9f"
server_key_version: "kuduclusterkey@0"
I20260812 03:04:53.845856 24236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "a5ee05a4153e497cbbc3c9619dadf760"
format_stamp: "Formatted at 2026-08-12 03:04:53 on dist-test-slave-6bbx"
server_key: "6db7307af6ab0185d92ebafed1015168"
server_key_iv: "b03b9c4b22e170a7ba0613c9c80f4d9f"
server_key_version: "kuduclusterkey@0"
I20260812 03:04:53.849870 24236 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 03:04:53.866956 24258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:53.868314 24236 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:04:53.868438 24236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "a5ee05a4153e497cbbc3c9619dadf760"
format_stamp: "Formatted at 2026-08-12 03:04:53 on dist-test-slave-6bbx"
server_key: "6db7307af6ab0185d92ebafed1015168"
server_key_iv: "b03b9c4b22e170a7ba0613c9c80f4d9f"
server_key_version: "kuduclusterkey@0"
I20260812 03:04:53.868548 24236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:04:53.885646 24236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:04:53.888330 24236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:04:53.888494 24236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:04:53.896570 24236 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:40143
I20260812 03:04:53.896639 24360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:40143 every 8 connection(s)
I20260812 03:04:53.897491 24236 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:04:53.901041 24361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:04:53.901125 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 24236
I20260812 03:04:53.901243 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:04:53.901477 5506 external_mini_cluster.cc:1428] Setting key 479d1a50dc812baff30490d4fb2b7b42
I20260812 03:04:53.909021 24361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5ee05a4153e497cbbc3c9619dadf760: Bootstrap starting.
Aug 12 03:04:53 dist-test-slave-6bbx krb5kdc[23354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:04:53.911566 24361 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5ee05a4153e497cbbc3c9619dadf760: Neither blocks nor log segments found. Creating new log.
I20260812 03:04:53.912931 24361 log.cc:824] T 00000000000000000000000000000000 P a5ee05a4153e497cbbc3c9619dadf760: Log is configured to *not* fsync() on all Append() calls
I20260812 03:04:53.914629 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:53.902786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52728 (local address 127.5.96.190:40143)
0812 03:04:53.903312 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:53.903327 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:53.903360 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:04:53.904295 (+ 935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:53.904315 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:53.904623 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:53.904911 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:53.905663 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:53.906616 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:53.907444 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:53.907943 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:53.908085 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:53.910333 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:04:53.910338 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:04:53.910350 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:04:53.910375 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:04:53.911860 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:04:53.912257 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:04:53.912262 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:04:53.912266 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:04:53.912364 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:04:53.912615 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:04:53.912619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:04:53.912621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:04:53.912977 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:04:53.913660 (+ 683us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:53.913794 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 03:04:53.913989 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":212,"threads_started":1}
I20260812 03:04:53.915145 24361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5ee05a4153e497cbbc3c9619dadf760: No bootstrap required, opened a new log
I20260812 03:04:53.919073 24361 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5ee05a4153e497cbbc3c9619dadf760 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5ee05a4153e497cbbc3c9619dadf760" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40143 } }
I20260812 03:04:53.919337 24361 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5ee05a4153e497cbbc3c9619dadf760 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:04:53.919397 24361 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5ee05a4153e497cbbc3c9619dadf760 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5ee05a4153e497cbbc3c9619dadf760, State: Initialized, Role: FOLLOWER
I20260812 03:04:53.919878 24361 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5ee05a4153e497cbbc3c9619dadf760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5ee05a4153e497cbbc3c9619dadf760" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40143 } }
I20260812 03:04:53.920046 24361 raft_consensus.cc:388] T 00000000000000000000000000000000 P a5ee05a4153e497cbbc3c9619dadf760 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:04:53.920115 24361 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5ee05a4153e497cbbc3c9619dadf760 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:04:53.920212 24361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5ee05a4153e497cbbc3c9619dadf760 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:04:53.921204 24361 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5ee05a4153e497cbbc3c9619dadf760 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5ee05a4153e497cbbc3c9619dadf760" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40143 } }
I20260812 03:04:53.921552 24361 leader_election.cc:302] T 00000000000000000000000000000000 P a5ee05a4153e497cbbc3c9619dadf760 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a5ee05a4153e497cbbc3c9619dadf760; no voters:
I20260812 03:04:53.921890 24361 leader_election.cc:288] T 00000000000000000000000000000000 P a5ee05a4153e497cbbc3c9619dadf760 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:04:53.921967 24378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5ee05a4153e497cbbc3c9619dadf760 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:04:53.922224 24378 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5ee05a4153e497cbbc3c9619dadf760 [term 1 LEADER]: Becoming Leader. State: Replica: a5ee05a4153e497cbbc3c9619dadf760, State: Running, Role: LEADER
I20260812 03:04:53.922506 24378 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5ee05a4153e497cbbc3c9619dadf760 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5ee05a4153e497cbbc3c9619dadf760" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40143 } }
I20260812 03:04:53.923106 24361 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5ee05a4153e497cbbc3c9619dadf760 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:04:53.924242 24379 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5ee05a4153e497cbbc3c9619dadf760 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5ee05a4153e497cbbc3c9619dadf760" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5ee05a4153e497cbbc3c9619dadf760" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40143 } } }
I20260812 03:04:53.924368 24379 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5ee05a4153e497cbbc3c9619dadf760 [sys.catalog]: This master's current role is: LEADER
I20260812 03:04:53.924605 24379 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5ee05a4153e497cbbc3c9619dadf760 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5ee05a4153e497cbbc3c9619dadf760. Latest consensus state: current_term: 1 leader_uuid: "a5ee05a4153e497cbbc3c9619dadf760" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5ee05a4153e497cbbc3c9619dadf760" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40143 } } }
I20260812 03:04:53.924690 24379 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5ee05a4153e497cbbc3c9619dadf760 [sys.catalog]: This master's current role is: LEADER
I20260812 03:04:53.925400 24361 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 03:04:53.929329 24391 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:04:53.933863 24391 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:04:53.942261 24391 catalog_manager.cc:1347] Generated new cluster ID: 308da6bbf9bd4e72ab1b2d0ad9054611
I20260812 03:04:53.942355 24391 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:04:53.955632 24391 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:04:53.956822 24391 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:04:53.960884 24391 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5ee05a4153e497cbbc3c9619dadf760: Generated new TSK 0
I20260812 03:04:53.961544 24391 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.96.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
I20260812 03:04:54.618868 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.96.148:55607/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:0
--local_ip_for_outbound_sockets=127.5.96.129
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:40143
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:04:54.738498 24447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:04:54.738868 24447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:04:54.738943 24447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:04:54.743203 24447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:04:54.743263 24447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:04:54.743341 24447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:04:54.748728 24447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.96.148:55607/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:40143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:04:54.749730 24447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:04:54.750743 24447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:04:54.760818 24454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:54.761242 24456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:54.761330 24458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:04:54.761766 24447 server_base.cc:1034] running on GCE node
I20260812 03:04:54.762226 24447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:04:54.762971 24447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:04:54.764165 24447 hybrid_clock.cc:648] HybridClock initialized: now 1786503894764140 us; error 37 us; skew 500 ppm
Aug 12 03:04:54 dist-test-slave-6bbx krb5kdc[23354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503894, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:04:54.768046 24447 init.cc:377] Logged in from keytab as kudu/127.5.96.129@KRBTEST.COM (short username kudu)
I20260812 03:04:54.768946 24447 webserver.cc:460] Webserver started at http://127.5.96.129:46441/ using document root <none> and password file <none>
I20260812 03:04:54.769505 24447 fs_manager.cc:359] Metadata directory not provided
I20260812 03:04:54.769573 24447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:04:54.769881 24447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:04:54 dist-test-slave-6bbx krb5kdc[23354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503894, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for HTTP/127.5.96.148@KRBTEST.COM
I20260812 03:04:54.787108 24447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-0/data/instance:
uuid: "19a5723d53e247e1ad2a34ade375135c"
format_stamp: "Formatted at 2026-08-12 03:04:54 on dist-test-slave-6bbx"
server_key: "7dcec4d193bed39ce7b08f0331faaf38"
server_key_iv: "d4381d6c913f5d8ea361d719e8f8f97b"
server_key_version: "kuduclusterkey@0"
I20260812 03:04:54.788015 24447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance:
uuid: "19a5723d53e247e1ad2a34ade375135c"
format_stamp: "Formatted at 2026-08-12 03:04:54 on dist-test-slave-6bbx"
server_key: "7dcec4d193bed39ce7b08f0331faaf38"
server_key_iv: "d4381d6c913f5d8ea361d719e8f8f97b"
server_key_version: "kuduclusterkey@0"
I20260812 03:04:54.794863 24447 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.001s
I20260812 03:04:54.816918 24470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:54.818730 24447 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.004s sys 0.000s
I20260812 03:04:54.818866 24447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "19a5723d53e247e1ad2a34ade375135c"
format_stamp: "Formatted at 2026-08-12 03:04:54 on dist-test-slave-6bbx"
server_key: "7dcec4d193bed39ce7b08f0331faaf38"
server_key_iv: "d4381d6c913f5d8ea361d719e8f8f97b"
server_key_version: "kuduclusterkey@0"
I20260812 03:04:54.818996 24447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:04:54.845235 24447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:04:54.848773 24447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:04:54.849013 24447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:04:54.849730 24447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:04:54.850807 24447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:04:54.850914 24447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:54.851032 24447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:04:54.851114 24447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:54.862069 24447 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:36021
I20260812 03:04:54.863274 24447 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:04:54.864198 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 24447
I20260812 03:04:54.864327 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance
I20260812 03:04:54.864632 5506 external_mini_cluster.cc:1428] Setting key 57e4eefbb994f9b6cd9aa5291bd08512
I20260812 03:04:54.880765 24684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:36021 every 8 connection(s)
WARNING: no policy specified for kudu/127.5.96.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.130@KRBTEST.COM" created.
Aug 12 03:04:54 dist-test-slave-6bbx krb5kdc[23354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503894, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:04:54.900887 24699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:54.884078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:48629 (local address 127.5.96.190:40143)
0812 03:04:54.884363 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:54.884371 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:54.885449 (+ 1078us) server_negotiation.cc:407] Connection header received
0812 03:04:54.887881 (+ 2432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:54.887886 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:54.887952 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:54.888023 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:54.890267 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:54.891077 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:54.892859 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:54.893201 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:54.893317 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:54.896324 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:04:54.896328 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:04:54.896331 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:04:54.896379 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:04:54.898260 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:04:54.899057 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:04:54.899060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:04:54.899062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:04:54.899172 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:04:54.899591 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:04:54.899595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:04:54.899597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:04:54.899873 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:04:54.899960 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:54.900589 (+ 629us) server_negotiation.cc:299] Negotiation successful
0812 03:04:54.900720 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":103,"threads_started":1}
I20260812 03:04:54.901803 24700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:54.884596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:40143 (local address 127.5.96.129:48629)
0812 03:04:54.885242 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:54.885312 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:54.886556 (+ 1244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:54.889071 (+ 2515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:54.889082 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:54.889409 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:54.890092 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:54.890113 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:54.891241 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:54.891244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:54.892705 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:54.892717 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:54.893507 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:54.893511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:54.893727 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:54.894415 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:54.894437 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:54.896126 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:04:54.898403 (+ 2277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:04:54.898409 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:04:54.898424 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:04:54.898927 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:04:54.899290 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:04:54.899293 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:04:54.899295 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:04:54.899477 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:04:54.900036 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:04:54.900042 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:04:54.900335 (+ 293us) client_negotiation.cc:769] Sending connection context
0812 03:04:54.900501 (+ 166us) client_negotiation.cc:240] Negotiation successful
0812 03:04:54.900811 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":145,"threads_started":1}
I20260812 03:04:54.903160 24686 heartbeater.cc:347] Connected to a master server at 127.5.96.190:40143
I20260812 03:04:54.903409 24686 heartbeater.cc:464] Registering TS with master...
I20260812 03:04:54.903980 24686 heartbeater.cc:511] Master 127.5.96.190:40143 requested a full tablet report, sending...
I20260812 03:04:54.905737 24292 ts_manager.cc:194] Registered new tserver with Master: 19a5723d53e247e1ad2a34ade375135c (127.5.96.129:36021)
I20260812 03:04:54.907307 24292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.129@KRBTEST.COM'} at 127.5.96.129:48629
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
I20260812 03:04:54.918145 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.96.148:55607/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:0
--local_ip_for_outbound_sockets=127.5.96.130
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:40143
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:04:54 dist-test-slave-6bbx krb5kdc[23354](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786503894, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 03:04:55.094302 24703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:04:55.095019 24703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:04:55.095176 24703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:04:55.100960 24703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:04:55.101079 24703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:04:55.101212 24703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:04:55.108948 24703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.96.148:55607/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:40143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:04:55.110527 24703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:04:55.111985 24703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:04:55.124763 24724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:55.124768 24728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:55.127463 24726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:04:55.127818 24703 server_base.cc:1034] running on GCE node
I20260812 03:04:55.128540 24703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:04:55.129397 24703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:04:55.130599 24703 hybrid_clock.cc:648] HybridClock initialized: now 1786503895130573 us; error 36 us; skew 500 ppm
Aug 12 03:04:55 dist-test-slave-6bbx krb5kdc[23354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503895, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:04:55.134397 24703 init.cc:377] Logged in from keytab as kudu/127.5.96.130@KRBTEST.COM (short username kudu)
I20260812 03:04:55.135462 24703 webserver.cc:460] Webserver started at http://127.5.96.130:44611/ using document root <none> and password file <none>
I20260812 03:04:55.136179 24703 fs_manager.cc:359] Metadata directory not provided
I20260812 03:04:55.136269 24703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:04:55.136503 24703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:04:55 dist-test-slave-6bbx krb5kdc[23354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503895, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for HTTP/127.5.96.148@KRBTEST.COM
I20260812 03:04:55.158753 24703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-1/data/instance:
uuid: "74c45120a2054065a7ab67371e1cd4e0"
format_stamp: "Formatted at 2026-08-12 03:04:55 on dist-test-slave-6bbx"
server_key: "979351de348b600d330469eb89d34697"
server_key_iv: "ef16f17230a6be88c94593c56928c225"
server_key_version: "kuduclusterkey@0"
I20260812 03:04:55.159497 24703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance:
uuid: "74c45120a2054065a7ab67371e1cd4e0"
format_stamp: "Formatted at 2026-08-12 03:04:55 on dist-test-slave-6bbx"
server_key: "979351de348b600d330469eb89d34697"
server_key_iv: "ef16f17230a6be88c94593c56928c225"
server_key_version: "kuduclusterkey@0"
I20260812 03:04:55.164674 24703 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 03:04:55.191917 24748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:55.193686 24703 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.000s
I20260812 03:04:55.193887 24703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "74c45120a2054065a7ab67371e1cd4e0"
format_stamp: "Formatted at 2026-08-12 03:04:55 on dist-test-slave-6bbx"
server_key: "979351de348b600d330469eb89d34697"
server_key_iv: "ef16f17230a6be88c94593c56928c225"
server_key_version: "kuduclusterkey@0"
I20260812 03:04:55.194110 24703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:04:55.220546 24703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:04:55.224128 24703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:04:55.224362 24703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:04:55.225113 24703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:04:55.226167 24703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:04:55.226272 24703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:55.226369 24703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:04:55.226428 24703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:55.246798 24703 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:45251
I20260812 03:04:55.248052 24703 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:04:55.248704 24970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:45251 every 8 connection(s)
I20260812 03:04:55.250622 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 24703
I20260812 03:04:55.250762 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance
I20260812 03:04:55.251099 5506 external_mini_cluster.cc:1428] Setting key bdb97bf41ea14a27192e43c1a3f96cbd
WARNING: no policy specified for kudu/127.5.96.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.131@KRBTEST.COM" created.
Aug 12 03:04:55 dist-test-slave-6bbx krb5kdc[23354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503895, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:04:55.278483 24699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:55.251221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:54121 (local address 127.5.96.190:40143)
0812 03:04:55.251372 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:55.251377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:55.258721 (+ 7344us) server_negotiation.cc:407] Connection header received
0812 03:04:55.260208 (+ 1487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:55.260213 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:55.260270 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:55.260349 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:55.262146 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.262998 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.268252 (+ 5254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.268642 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.268762 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:55.274029 (+ 5267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:04:55.274034 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:04:55.274037 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:04:55.274065 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:04:55.275844 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:04:55.276509 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:04:55.276513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:04:55.276515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:04:55.276613 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:04:55.277040 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:04:55.277043 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:04:55.277045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:04:55.277281 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:04:55.277387 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:55.278110 (+ 723us) server_negotiation.cc:299] Negotiation successful
0812 03:04:55.278259 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 03:04:55.280426 24975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:55.251749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:40143 (local address 127.5.96.130:54121)
0812 03:04:55.258553 (+ 6804us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:55.258614 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:55.259911 (+ 1297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:55.260692 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:55.260703 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:55.261214 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:55.261960 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:55.261983 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.266429 (+ 4446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.266433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:55.268125 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:55.268138 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.268829 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.268834 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:55.271232 (+ 2398us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:55.272060 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:55.272081 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:55.273849 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:04:55.275986 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:04:55.275993 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:04:55.276008 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:04:55.276378 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:04:55.276732 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:04:55.276735 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:04:55.276737 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:04:55.276920 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:04:55.277464 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:04:55.277470 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:04:55.277796 (+ 326us) client_negotiation.cc:769] Sending connection context
0812 03:04:55.278941 (+ 1145us) client_negotiation.cc:240] Negotiation successful
0812 03:04:55.279286 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6533,"thread_start_us":148,"threads_started":1}
I20260812 03:04:55.281992 24973 heartbeater.cc:347] Connected to a master server at 127.5.96.190:40143
I20260812 03:04:55.282258 24973 heartbeater.cc:464] Registering TS with master...
I20260812 03:04:55.282889 24973 heartbeater.cc:511] Master 127.5.96.190:40143 requested a full tablet report, sending...
I20260812 03:04:55.284159 24292 ts_manager.cc:194] Registered new tserver with Master: 74c45120a2054065a7ab67371e1cd4e0 (127.5.96.130:45251)
I20260812 03:04:55.284925 24292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.130@KRBTEST.COM'} at 127.5.96.130:54121
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
I20260812 03:04:55.304747 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.96.148:55607/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:0
--local_ip_for_outbound_sockets=127.5.96.131
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:40143
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:04:55.491180 24980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:04:55.491554 24980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:04:55.491624 24980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:04:55.497699 24980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:04:55.497776 24980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:04:55.497856 24980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:04:55.505429 24980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.96.148:55607/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:40143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:04:55.506726 24980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:04:55.508059 24980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:04:55.521332 24987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:55.521301 24988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:04:55.521739 24990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:04:55.522806 24980 server_base.cc:1034] running on GCE node
I20260812 03:04:55.523315 24980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:04:55.524045 24980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:04:55.525229 24980 hybrid_clock.cc:648] HybridClock initialized: now 1786503895525157 us; error 83 us; skew 500 ppm
Aug 12 03:04:55 dist-test-slave-6bbx krb5kdc[23354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503895, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:04:55.528676 24980 init.cc:377] Logged in from keytab as kudu/127.5.96.131@KRBTEST.COM (short username kudu)
I20260812 03:04:55.529722 24980 webserver.cc:460] Webserver started at http://127.5.96.131:44227/ using document root <none> and password file <none>
I20260812 03:04:55.530423 24980 fs_manager.cc:359] Metadata directory not provided
I20260812 03:04:55.530506 24980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:04:55.530722 24980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:04:55 dist-test-slave-6bbx krb5kdc[23354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503895, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for HTTP/127.5.96.148@KRBTEST.COM
I20260812 03:04:55.551275 24980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-2/data/instance:
uuid: "c6af67004e8646eab6c39dd2618c099f"
format_stamp: "Formatted at 2026-08-12 03:04:55 on dist-test-slave-6bbx"
server_key: "e2e740d3f541f72999de606a254b47e0"
server_key_iv: "b67b5036259b0155ba473ac239980da3"
server_key_version: "kuduclusterkey@0"
I20260812 03:04:55.551981 24980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance:
uuid: "c6af67004e8646eab6c39dd2618c099f"
format_stamp: "Formatted at 2026-08-12 03:04:55 on dist-test-slave-6bbx"
server_key: "e2e740d3f541f72999de606a254b47e0"
server_key_iv: "b67b5036259b0155ba473ac239980da3"
server_key_version: "kuduclusterkey@0"
I20260812 03:04:55.557209 24980 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260812 03:04:55.584013 25003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
Aug 12 03:04:55 dist-test-slave-6bbx krb5kdc[23354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503894, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for HTTP/127.5.96.148@KRBTEST.COM
I20260812 03:04:55.593014 24980 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.003s sys 0.000s
I20260812 03:04:55.593181 24980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "c6af67004e8646eab6c39dd2618c099f"
format_stamp: "Formatted at 2026-08-12 03:04:55 on dist-test-slave-6bbx"
server_key: "e2e740d3f541f72999de606a254b47e0"
server_key_iv: "b67b5036259b0155ba473ac239980da3"
server_key_version: "kuduclusterkey@0"
I20260812 03:04:55.593336 24980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:04:55.628302 24980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:04:55.631907 24980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:04:55.632091 24980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:04:55.632792 24980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:04:55.633860 24980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:04:55.633925 24980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:55.633981 24980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:04:55.634008 24980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:04:55.647145 24980 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:38411
I20260812 03:04:55.648257 24980 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:04:55.648468 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 24980
I20260812 03:04:55.648593 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance
I20260812 03:04:55.648906 5506 external_mini_cluster.cc:1428] Setting key c8cd6af9df6bdd03b3f44a400f616dca
I20260812 03:04:55.664054 25214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:38411 every 8 connection(s)
Aug 12 03:04:55 dist-test-slave-6bbx krb5kdc[23354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503895, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:04:55.698567 24699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:55.667019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:53371 (local address 127.5.96.190:40143)
0812 03:04:55.667144 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:55.667148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:55.674099 (+ 6951us) server_negotiation.cc:407] Connection header received
0812 03:04:55.685130 (+ 11031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:55.685134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:55.685192 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:55.685242 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:55.687325 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.688201 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.690127 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.690475 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.690545 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:55.693801 (+ 3256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:04:55.693804 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:04:55.693807 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:04:55.693835 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:04:55.695745 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:04:55.696572 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:04:55.696576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:04:55.696578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:04:55.696675 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:04:55.697262 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:04:55.697265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:04:55.697267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:04:55.697475 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:04:55.697546 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:55.698288 (+ 742us) server_negotiation.cc:299] Negotiation successful
0812 03:04:55.698404 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:04:55.700553 25218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:55.667585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:40143 (local address 127.5.96.131:53371)
0812 03:04:55.673929 (+ 6344us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:55.673996 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:55.684841 (+ 10845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:55.685905 (+ 1064us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:55.685916 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:55.686376 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:04:55.687154 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:55.687175 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.688359 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.688362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:55.689985 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:55.689997 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.690763 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.690766 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:55.691041 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:55.691844 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:04:55.691865 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:04:55.693617 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:04:55.695884 (+ 2267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:04:55.695889 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:04:55.695905 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:04:55.696445 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:04:55.696784 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:04:55.696787 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:04:55.696789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:04:55.697147 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:04:55.697709 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:04:55.697715 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:04:55.698014 (+ 299us) client_negotiation.cc:769] Sending connection context
0812 03:04:55.699048 (+ 1034us) client_negotiation.cc:240] Negotiation successful
0812 03:04:55.699373 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5256,"thread_start_us":271,"threads_started":1}
I20260812 03:04:55.706087 25215 heartbeater.cc:347] Connected to a master server at 127.5.96.190:40143
I20260812 03:04:55.706377 25215 heartbeater.cc:464] Registering TS with master...
I20260812 03:04:55.707039 25215 heartbeater.cc:511] Master 127.5.96.190:40143 requested a full tablet report, sending...
I20260812 03:04:55.708191 24292 ts_manager.cc:194] Registered new tserver with Master: c6af67004e8646eab6c39dd2618c099f (127.5.96.131:38411)
I20260812 03:04:55.708894 24292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.131@KRBTEST.COM'} at 127.5.96.131:53371
I20260812 03:04:55.709263 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:04:55.721203 24699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:55.711583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52742 (local address 127.5.96.190:40143)
0812 03:04:55.711743 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:55.711748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:55.712304 (+ 556us) server_negotiation.cc:407] Connection header received
0812 03:04:55.712477 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:55.712480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:55.712524 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:55.712568 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:04:55.713473 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.714274 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.715492 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.715888 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.715967 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:55.717704 (+ 1737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:04:55.717708 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:04:55.717711 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:04:55.717736 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:04:55.719306 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:04:55.719824 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:04:55.719827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:04:55.719829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:04:55.719914 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:04:55.720230 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:04:55.720233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:04:55.720235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:04:55.720441 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:04:55.720550 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:55.720933 (+ 383us) server_negotiation.cc:299] Negotiation successful
0812 03:04:55.721048 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 03:04:55.724998 24292 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52742:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:04:55.729631 24292 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:04:55.749296 25239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:55.742276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:38411 (local address 127.0.0.1:43608)
0812 03:04:55.743816 (+ 1540us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:55.743828 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:55.743923 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:55.744582 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:55.744588 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:55.744611 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:04:55.744761 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:55.744769 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.746126 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.746129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:55.747458 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:55.747469 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.748764 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.748768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:55.748877 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:55.748919 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 03:04:55.749014 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 03:04:55.749147 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1479,"mutex_wait_us":21,"thread_start_us":91,"threads_started":1}
I20260812 03:04:55.749890 25249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:55.742704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43608 (local address 127.5.96.131:38411)
0812 03:04:55.743632 (+ 928us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:55.743647 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:55.743920 (+ 273us) server_negotiation.cc:407] Connection header received
0812 03:04:55.744032 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:55.744038 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:55.744264 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:55.744413 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:04:55.744895 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.745989 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.747611 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.748457 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.748554 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:55.748700 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:55.749650 (+ 950us) server_negotiation.cc:299] Negotiation successful
0812 03:04:55.749747 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":809,"thread_start_us":143,"threads_started":1}
I20260812 03:04:55.750666 25246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:55.740553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:36021 (local address 127.0.0.1:33822)
0812 03:04:55.742652 (+ 2099us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:55.742700 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:55.742828 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:55.744696 (+ 1868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:55.744699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:55.744713 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:04:55.744813 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:55.744819 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.746919 (+ 2100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.746922 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:55.748032 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:55.748042 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.749215 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.749219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:55.750373 (+ 1154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:55.750399 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 03:04:55.750463 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 03:04:55.750530 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1924,"mutex_wait_us":4,"thread_start_us":116,"threads_started":1}
I20260812 03:04:55.751230 25242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:55.741406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33822 (local address 127.5.96.129:36021)
0812 03:04:55.744162 (+ 2756us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:55.744169 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:55.744190 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:04:55.744258 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:55.744262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:55.744425 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:55.744549 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:04:55.744928 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.746791 (+ 1863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.748166 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.748918 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.749018 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:55.749154 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:55.750993 (+ 1839us) server_negotiation.cc:299] Negotiation successful
0812 03:04:55.751092 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2654,"thread_start_us":70,"threads_started":1}
I20260812 03:04:55.753717 25086 tablet_service.cc:1467] Processing CreateTablet for tablet 68390aba1ec34ddb8205bb0b0e71f8b3 (DEFAULT_TABLE table=test-table [id=5286a429265e4094afda92a20f1fc7b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:04:55.756608 24556 tablet_service.cc:1467] Processing CreateTablet for tablet 68390aba1ec34ddb8205bb0b0e71f8b3 (DEFAULT_TABLE table=test-table [id=5286a429265e4094afda92a20f1fc7b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:04:55.757782 24556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68390aba1ec34ddb8205bb0b0e71f8b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:04:55.758551 25239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:55.743465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:45251 (local address 127.0.0.1:34276)
0812 03:04:55.749451 (+ 5986us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:55.749463 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:55.749531 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:55.752941 (+ 3410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:55.752945 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:55.752961 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:04:55.753032 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:55.753037 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.755431 (+ 2394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.755434 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:55.756548 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:55.756561 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.757974 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.757978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:55.758272 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:55.758297 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 03:04:55.758368 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 03:04:55.758431 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5934,"thread_start_us":73,"threads_started":1}
I20260812 03:04:55.759053 25251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:55.743871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34276 (local address 127.5.96.130:45251)
0812 03:04:55.752303 (+ 8432us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:55.752325 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:55.752348 (+ 23us) server_negotiation.cc:407] Connection header received
0812 03:04:55.752420 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:55.752424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:55.752632 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:55.752789 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:04:55.753365 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.755300 (+ 1935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.756773 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.757666 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.757767 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:55.757908 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:55.758739 (+ 831us) server_negotiation.cc:299] Negotiation successful
0812 03:04:55.758844 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8325,"thread_start_us":75,"threads_started":1}
I20260812 03:04:55.759581 25086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68390aba1ec34ddb8205bb0b0e71f8b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:04:55.762173 24836 tablet_service.cc:1467] Processing CreateTablet for tablet 68390aba1ec34ddb8205bb0b0e71f8b3 (DEFAULT_TABLE table=test-table [id=5286a429265e4094afda92a20f1fc7b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:04:55.763190 24836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68390aba1ec34ddb8205bb0b0e71f8b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:04:55.768755 25252 tablet_bootstrap.cc:492] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f: Bootstrap starting.
I20260812 03:04:55.771934 25252 tablet_bootstrap.cc:654] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f: Neither blocks nor log segments found. Creating new log.
I20260812 03:04:55.773067 25253 tablet_bootstrap.cc:492] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 74c45120a2054065a7ab67371e1cd4e0: Bootstrap starting.
I20260812 03:04:55.773051 25252 log.cc:824] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:04:55.775146 25252 tablet_bootstrap.cc:492] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f: No bootstrap required, opened a new log
I20260812 03:04:55.775341 25252 ts_tablet_manager.cc:1397] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 03:04:55.776923 25253 tablet_bootstrap.cc:654] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 74c45120a2054065a7ab67371e1cd4e0: Neither blocks nor log segments found. Creating new log.
I20260812 03:04:55.777878 25253 log.cc:824] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 74c45120a2054065a7ab67371e1cd4e0: Log is configured to *not* fsync() on all Append() calls
I20260812 03:04:55.778409 25254 tablet_bootstrap.cc:492] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 19a5723d53e247e1ad2a34ade375135c: Bootstrap starting.
I20260812 03:04:55.779686 25252 raft_consensus.cc:348] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19a5723d53e247e1ad2a34ade375135c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36021 } } peers { permanent_uuid: "c6af67004e8646eab6c39dd2618c099f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38411 } } peers { permanent_uuid: "74c45120a2054065a7ab67371e1cd4e0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 45251 } }
I20260812 03:04:55.779976 25252 raft_consensus.cc:374] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:04:55.780030 25252 raft_consensus.cc:729] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6af67004e8646eab6c39dd2618c099f, State: Initialized, Role: FOLLOWER
I20260812 03:04:55.780524 25252 consensus_queue.cc:260] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19a5723d53e247e1ad2a34ade375135c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36021 } } peers { permanent_uuid: "c6af67004e8646eab6c39dd2618c099f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38411 } } peers { permanent_uuid: "74c45120a2054065a7ab67371e1cd4e0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 45251 } }
I20260812 03:04:55.781299 25254 tablet_bootstrap.cc:654] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 19a5723d53e247e1ad2a34ade375135c: Neither blocks nor log segments found. Creating new log.
I20260812 03:04:55.781635 25252 ts_tablet_manager.cc:1428] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 03:04:55.782030 25215 heartbeater.cc:503] Master 127.5.96.190:40143 was elected leader, sending a full tablet report...
I20260812 03:04:55.782634 25254 log.cc:824] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 19a5723d53e247e1ad2a34ade375135c: Log is configured to *not* fsync() on all Append() calls
I20260812 03:04:55.785444 25253 tablet_bootstrap.cc:492] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 74c45120a2054065a7ab67371e1cd4e0: No bootstrap required, opened a new log
I20260812 03:04:55.785660 25253 ts_tablet_manager.cc:1397] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 74c45120a2054065a7ab67371e1cd4e0: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.000s
I20260812 03:04:55.789883 25253 raft_consensus.cc:348] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 74c45120a2054065a7ab67371e1cd4e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19a5723d53e247e1ad2a34ade375135c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36021 } } peers { permanent_uuid: "c6af67004e8646eab6c39dd2618c099f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38411 } } peers { permanent_uuid: "74c45120a2054065a7ab67371e1cd4e0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 45251 } }
I20260812 03:04:55.790171 25253 raft_consensus.cc:374] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 74c45120a2054065a7ab67371e1cd4e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:04:55.790238 25253 raft_consensus.cc:729] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 74c45120a2054065a7ab67371e1cd4e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74c45120a2054065a7ab67371e1cd4e0, State: Initialized, Role: FOLLOWER
I20260812 03:04:55.790831 25253 consensus_queue.cc:260] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 74c45120a2054065a7ab67371e1cd4e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19a5723d53e247e1ad2a34ade375135c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36021 } } peers { permanent_uuid: "c6af67004e8646eab6c39dd2618c099f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38411 } } peers { permanent_uuid: "74c45120a2054065a7ab67371e1cd4e0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 45251 } }
I20260812 03:04:55.791827 25253 ts_tablet_manager.cc:1428] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 74c45120a2054065a7ab67371e1cd4e0: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 03:04:55.792153 24973 heartbeater.cc:503] Master 127.5.96.190:40143 was elected leader, sending a full tablet report...
I20260812 03:04:55.793339 25254 tablet_bootstrap.cc:492] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 19a5723d53e247e1ad2a34ade375135c: No bootstrap required, opened a new log
I20260812 03:04:55.793542 25254 ts_tablet_manager.cc:1397] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 19a5723d53e247e1ad2a34ade375135c: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.005s
I20260812 03:04:55.798341 25254 raft_consensus.cc:348] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 19a5723d53e247e1ad2a34ade375135c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19a5723d53e247e1ad2a34ade375135c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36021 } } peers { permanent_uuid: "c6af67004e8646eab6c39dd2618c099f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38411 } } peers { permanent_uuid: "74c45120a2054065a7ab67371e1cd4e0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 45251 } }
I20260812 03:04:55.798651 25254 raft_consensus.cc:374] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 19a5723d53e247e1ad2a34ade375135c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:04:55.798718 25254 raft_consensus.cc:729] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 19a5723d53e247e1ad2a34ade375135c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19a5723d53e247e1ad2a34ade375135c, State: Initialized, Role: FOLLOWER
I20260812 03:04:55.799300 25254 consensus_queue.cc:260] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 19a5723d53e247e1ad2a34ade375135c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19a5723d53e247e1ad2a34ade375135c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36021 } } peers { permanent_uuid: "c6af67004e8646eab6c39dd2618c099f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38411 } } peers { permanent_uuid: "74c45120a2054065a7ab67371e1cd4e0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 45251 } }
I20260812 03:04:55.800473 25254 ts_tablet_manager.cc:1428] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 19a5723d53e247e1ad2a34ade375135c: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 03:04:55.800853 24686 heartbeater.cc:503] Master 127.5.96.190:40143 was elected leader, sending a full tablet report...
W20260812 03:04:55.884634 24688 tablet.cc:2369] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 19a5723d53e247e1ad2a34ade375135c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:04:55.915609 25217 tablet.cc:2369] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:04:55.924389 25259 raft_consensus.cc:482] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:04:55.924618 25259 raft_consensus.cc:504] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19a5723d53e247e1ad2a34ade375135c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36021 } } peers { permanent_uuid: "c6af67004e8646eab6c39dd2618c099f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38411 } } peers { permanent_uuid: "74c45120a2054065a7ab67371e1cd4e0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 45251 } }
I20260812 03:04:55.926096 25259 leader_election.cc:288] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19a5723d53e247e1ad2a34ade375135c (127.5.96.129:36021), 74c45120a2054065a7ab67371e1cd4e0 (127.5.96.130:45251)
I20260812 03:04:55.931295 25218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:55.926279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:36021 (local address 127.5.96.131:56521)
0812 03:04:55.926474 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:55.926492 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:55.926595 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:55.927377 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:55.927380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:55.927403 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:04:55.927541 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:55.927548 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.928660 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.928663 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:55.929867 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:55.929878 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.930881 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.930884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:55.930981 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:55.931004 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:04:55.931073 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 03:04:55.931161 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260812 03:04:55.932284 25242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:55.926916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:56521 (local address 127.5.96.129:36021)
0812 03:04:55.927139 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:55.927144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:55.927159 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:04:55.927209 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:55.927213 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:55.927260 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:55.927333 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:04:55.927737 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.928522 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.930010 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.930587 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.930696 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:55.930747 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:55.932070 (+ 1323us) server_negotiation.cc:299] Negotiation successful
0812 03:04:55.932166 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260812 03:04:55.932842 24561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68390aba1ec34ddb8205bb0b0e71f8b3" candidate_uuid: "c6af67004e8646eab6c39dd2618c099f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19a5723d53e247e1ad2a34ade375135c" is_pre_election: true
I20260812 03:04:55.933174 24561 raft_consensus.cc:2454] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 19a5723d53e247e1ad2a34ade375135c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6af67004e8646eab6c39dd2618c099f in term 0.
I20260812 03:04:55.933686 25007 leader_election.cc:302] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19a5723d53e247e1ad2a34ade375135c, c6af67004e8646eab6c39dd2618c099f; no voters:
I20260812 03:04:55.933974 25259 raft_consensus.cc:2781] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:04:55.934070 25259 raft_consensus.cc:482] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:04:55.934130 25259 raft_consensus.cc:3037] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:04:55.935547 25259 raft_consensus.cc:504] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19a5723d53e247e1ad2a34ade375135c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36021 } } peers { permanent_uuid: "c6af67004e8646eab6c39dd2618c099f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38411 } } peers { permanent_uuid: "74c45120a2054065a7ab67371e1cd4e0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 45251 } }
I20260812 03:04:55.935921 25259 leader_election.cc:288] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f [CANDIDATE]: Term 1 election: Requested vote from peers 19a5723d53e247e1ad2a34ade375135c (127.5.96.129:36021), 74c45120a2054065a7ab67371e1cd4e0 (127.5.96.130:45251)
I20260812 03:04:55.936489 24561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68390aba1ec34ddb8205bb0b0e71f8b3" candidate_uuid: "c6af67004e8646eab6c39dd2618c099f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19a5723d53e247e1ad2a34ade375135c"
I20260812 03:04:55.936621 24561 raft_consensus.cc:3037] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 19a5723d53e247e1ad2a34ade375135c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:04:55.937842 24561 raft_consensus.cc:2454] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 19a5723d53e247e1ad2a34ade375135c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6af67004e8646eab6c39dd2618c099f in term 1.
I20260812 03:04:55.939594 25218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:55.926950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:45251 (local address 127.5.96.131:40381)
0812 03:04:55.931464 (+ 4514us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:55.931479 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:04:55.931557 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:04:55.933497 (+ 1940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:04:55.933501 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:04:55.933521 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:04:55.933601 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:55.933607 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.934917 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.934920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:55.936073 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:04:55.936085 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.939125 (+ 3040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.939129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:04:55.939215 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:55.939240 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 03:04:55.939345 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 03:04:55.939416 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4459,"thread_start_us":104,"threads_started":1}
I20260812 03:04:55.940078 25007 leader_election.cc:302] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19a5723d53e247e1ad2a34ade375135c, c6af67004e8646eab6c39dd2618c099f; no voters:
I20260812 03:04:55.940261 25259 raft_consensus.cc:2781] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:04:55.940476 25259 raft_consensus.cc:686] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f [term 1 LEADER]: Becoming Leader. State: Replica: c6af67004e8646eab6c39dd2618c099f, State: Running, Role: LEADER
I20260812 03:04:55.940821 25259 consensus_queue.cc:237] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19a5723d53e247e1ad2a34ade375135c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36021 } } peers { permanent_uuid: "c6af67004e8646eab6c39dd2618c099f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38411 } } peers { permanent_uuid: "74c45120a2054065a7ab67371e1cd4e0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 45251 } }
I20260812 03:04:55.941284 25251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:55.927530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:40381 (local address 127.5.96.130:45251)
0812 03:04:55.927651 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:55.927655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:55.931635 (+ 3980us) server_negotiation.cc:407] Connection header received
0812 03:04:55.931724 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:55.931727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:55.931777 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:55.931844 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:04:55.934027 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.934783 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.938222 (+ 3439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.938859 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.938969 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:55.939019 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:55.941123 (+ 2104us) server_negotiation.cc:299] Negotiation successful
0812 03:04:55.941194 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:04:55.945616 24876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68390aba1ec34ddb8205bb0b0e71f8b3" candidate_uuid: "c6af67004e8646eab6c39dd2618c099f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74c45120a2054065a7ab67371e1cd4e0" is_pre_election: true
I20260812 03:04:55.945904 24876 raft_consensus.cc:2454] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 74c45120a2054065a7ab67371e1cd4e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6af67004e8646eab6c39dd2618c099f in term 0.
I20260812 03:04:55.946250 24872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68390aba1ec34ddb8205bb0b0e71f8b3" candidate_uuid: "c6af67004e8646eab6c39dd2618c099f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74c45120a2054065a7ab67371e1cd4e0"
I20260812 03:04:55.946374 24872 raft_consensus.cc:3037] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 74c45120a2054065a7ab67371e1cd4e0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:04:55.947599 24872 raft_consensus.cc:2454] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 74c45120a2054065a7ab67371e1cd4e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6af67004e8646eab6c39dd2618c099f in term 1.
I20260812 03:04:55.948400 24292 catalog_manager.cc:5495] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f reported cstate change: term changed from 0 to 1, leader changed from <none> to c6af67004e8646eab6c39dd2618c099f (127.5.96.131). New cstate: current_term: 1 leader_uuid: "c6af67004e8646eab6c39dd2618c099f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19a5723d53e247e1ad2a34ade375135c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6af67004e8646eab6c39dd2618c099f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 38411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c45120a2054065a7ab67371e1cd4e0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 45251 } health_report { overall_health: UNKNOWN } } }
Aug 12 03:04:55 dist-test-slave-6bbx krb5kdc[23354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503894, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for HTTP/127.5.96.148@KRBTEST.COM
I20260812 03:04:55.987244 25249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:55.982766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43622 (local address 127.5.96.131:38411)
0812 03:04:55.983051 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0812 03:04:55.983054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:04:55.983066 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:04:55.983139 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:04:55.983141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:04:55.983193 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:04:55.983287 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:04:55.983588 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.984273 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.985296 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:04:55.985627 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:04:55.985727 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:04:55.986053 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:04:55.986553 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:04:55.986688 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 03:04:55.987032 (+ 344us) server_negotiation.cc:299] Negotiation successful
0812 03:04:55.987098 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209}
I20260812 03:04:55.993860 24872 raft_consensus.cc:1261] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 74c45120a2054065a7ab67371e1cd4e0 [term 1 FOLLOWER]: Refusing update from remote peer c6af67004e8646eab6c39dd2618c099f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:04:55.994577 24561 raft_consensus.cc:1261] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 19a5723d53e247e1ad2a34ade375135c [term 1 FOLLOWER]: Refusing update from remote peer c6af67004e8646eab6c39dd2618c099f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:04:55.994616 25259 consensus_queue.cc:1035] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c45120a2054065a7ab67371e1cd4e0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 45251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:04:55.995254 25259 consensus_queue.cc:1035] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f [LEADER]: Connected to new peer: Peer: permanent_uuid: "19a5723d53e247e1ad2a34ade375135c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 03:04:55.999946 24974 tablet.cc:2369] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 74c45120a2054065a7ab67371e1cd4e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:04:56.004369 25279 mvcc.cc:205] Tried to move back new op lower bound from 7317519957983141888 to 7317519957776441344. Current Snapshot: MvccSnapshot[applied={T|T < 7317519957983141888}]
I20260812 03:04:56.013399 24292 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:04:56.013558 24292 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:04:56.016903 24292 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a5ee05a4153e497cbbc3c9619dadf760: Sending DeleteTablet for 3 replicas of tablet 68390aba1ec34ddb8205bb0b0e71f8b3
I20260812 03:04:56.017719 25086 tablet_service.cc:1514] Processing DeleteTablet for tablet 68390aba1ec34ddb8205bb0b0e71f8b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:04:56 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:43608
I20260812 03:04:56.017745 24836 tablet_service.cc:1514] Processing DeleteTablet for tablet 68390aba1ec34ddb8205bb0b0e71f8b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:04:56 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:34276
I20260812 03:04:56.017933 24556 tablet_service.cc:1514] Processing DeleteTablet for tablet 68390aba1ec34ddb8205bb0b0e71f8b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:04:56 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:33822
I20260812 03:04:56.018215 25292 tablet_replica.cc:330] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f: stopping tablet replica
I20260812 03:04:56.018522 25292 raft_consensus.cc:2229] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:04:56.018808 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 24447
I20260812 03:04:56.018842 25292 raft_consensus.cc:2258] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:04:56.020469 25292 ts_tablet_manager.cc:1905] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:04:56.022825 25292 ts_tablet_manager.cc:1918] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:04:56.022903 25292 log.cc:1196] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-2/wal/wals/68390aba1ec34ddb8205bb0b0e71f8b3
I20260812 03:04:56.023250 25292 ts_tablet_manager.cc:1939] T 68390aba1ec34ddb8205bb0b0e71f8b3 P c6af67004e8646eab6c39dd2618c099f: Deleting consensus metadata
I20260812 03:04:56.024259 24261 catalog_manager.cc:4841] TS c6af67004e8646eab6c39dd2618c099f (127.5.96.131:38411): tablet 68390aba1ec34ddb8205bb0b0e71f8b3 (table test-table [id=5286a429265e4094afda92a20f1fc7b4]) successfully deleted
I20260812 03:04:56.028185 25294 tablet_replica.cc:330] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 74c45120a2054065a7ab67371e1cd4e0: stopping tablet replica
I20260812 03:04:56.028484 25294 raft_consensus.cc:2229] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 74c45120a2054065a7ab67371e1cd4e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:04:56.028707 25294 raft_consensus.cc:2258] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 74c45120a2054065a7ab67371e1cd4e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:04:56.051134 25294 ts_tablet_manager.cc:1905] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 74c45120a2054065a7ab67371e1cd4e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:04:56.054309 25294 ts_tablet_manager.cc:1918] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 74c45120a2054065a7ab67371e1cd4e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:04:56.054550 25294 log.cc:1196] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 74c45120a2054065a7ab67371e1cd4e0: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786503790637001-5506-0/minicluster-data/ts-1/wal/wals/68390aba1ec34ddb8205bb0b0e71f8b3
I20260812 03:04:56.054935 25294 ts_tablet_manager.cc:1939] T 68390aba1ec34ddb8205bb0b0e71f8b3 P 74c45120a2054065a7ab67371e1cd4e0: Deleting consensus metadata
I20260812 03:04:56.056423 24262 catalog_manager.cc:4841] TS 74c45120a2054065a7ab67371e1cd4e0 (127.5.96.130:45251): tablet 68390aba1ec34ddb8205bb0b0e71f8b3 (table test-table [id=5286a429265e4094afda92a20f1fc7b4]) successfully deleted
W20260812 03:04:56.057821 24265 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.96.129:36021 (error 108)
I20260812 03:04:56.059276 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 24703
I20260812 03:04:56.059300 25239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:04:56.059009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:36021 (local address 127.0.0.1:33828)
0812 03:04:56.059127 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 03:04:56.059211 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.129:36021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 03:04:56.061774 24265 catalog_manager.cc:4584] TS 19a5723d53e247e1ad2a34ade375135c (127.5.96.129:36021): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 68390aba1ec34ddb8205bb0b0e71f8b3: Network error: Client connection negotiation failed: client connection to 127.5.96.129:36021: connect: Connection refused (error 111)
I20260812 03:04:56.080906 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 24980
I20260812 03:04:56.097055 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 24236
I20260812 03:04:56.114389 5506 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 03:04:56.469631 5506 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 03:04:56.469753 5506 mini_ranger.cc:66] Stopping Ranger...
I20260812 03:04:56.813174 5506 mini_ranger.cc:68] Stopped Ranger
2026-08-12 03:04:56.813 UTC [23425] LOG: received smart shutdown request
2026-08-12 03:04:56.817 UTC [23425] LOG: background worker "logical replication launcher" (PID 23444) exited with exit code 1
2026-08-12 03:04:56.817 UTC [23433] LOG: shutting down
2026-08-12 03:04:56.942 UTC [23425] LOG: database system is shut down
2026-08-12T03:04:56Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (43819 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:04:57 dist-test-slave-6bbx krb5kdc[25319](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:04:57 dist-test-slave-6bbx krb5kdc[25319](info): set up 2 sockets
Aug 12 03:04:57 dist-test-slave-6bbx krb5kdc[25319](info): commencing operation
krb5kdc: starting...
W20260812 03:04:58.185796 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:04:58 dist-test-slave-6bbx krb5kdc[25319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:04:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:04:58Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.96.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.96.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.96.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/rangeradmin_127.5.96.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.96.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/rangeradmin_127.5.96.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.96.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.96.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.96.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/rangerlookup_127.5.96.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.96.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/rangerlookup_127.5.96.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.96.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.96.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.96.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/HTTP_127.5.96.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.96.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/HTTP_127.5.96.148@KRBTEST.COM.keytab.
I20260812 03:04:58.335081 5506 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task6aqATZ/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/postgres -l logfile start
2026-08-12 03:05:00.391 UTC [25384] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 03:05:00.391 UTC [25384] LOG: listening on IPv4 address "127.5.96.148", port 36171
2026-08-12 03:05:00.393 UTC [25384] LOG: listening on Unix socket "/tmp/.s.PGSQL.36171"
2026-08-12 03:05:00.412 UTC [25388] LOG: database system was shut down at 2026-08-12 03:04:59 UTC
2026-08-12 03:05:00.415 UTC [25384] LOG: database system is ready to accept connections
I20260812 03:05:01.480441 5506 mini_postgres.cc:96] Postgres bound to 36171
2026-08-12 03:05:01.487 UTC [25404] FATAL: database "slave" does not exist
127.5.96.148:36171 - accepting connections
I20260812 03:05:01.488181 5506 mini_ranger.cc:161] Starting Ranger...
I20260812 03:05:01.502656 5506 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 03:05:01.832021 5506 mini_ranger.cc:87] Created ranger Postgres database
I20260812 03:05:01.832093 5506 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ranger-admin
2026-08-12 03:05:02,101 [I] DB FLAVOR :POSTGRES
2026-08-12 03:05:02,101 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 03:05:02,101 [I] Checking connection..
2026-08-12 03:05:02,102 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:36171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 03:05:02,323 [I] Checking connection passed.
2026-08-12 03:05:02,323 [I] --------- Verifying version history table ---------
2026-08-12 03:05:02,323 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:36171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:05:02,548 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 03:05:02,549 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:36171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:05:02,778 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 03:05:02,778 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 03:05:02,779 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:36171/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 03:05:03.027 UTC [25511] WARNING: there is no transaction in progress
2026-08-12 03:05:03,035 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 03:05:03,035 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:36171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:05:03,266 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 03:05:03,266 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 03:05:03,267 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:36171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 03:05:03,505 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:36171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 03:05:03,755 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:36171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6bbx.c.gcp-upstream.internal','N') ;"
2026-08-12 03:05:03,969 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:36171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 03:05:04,223 [I] Table x_portal_user does not exist in database ranger
2026-08-12 03:05:04,223 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:36171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 03:05:04,451 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 03:05:04,451 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:36171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 03:05:04,701 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:36171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 03:05:04,951 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 03:05:04,951 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:36171/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 03:05:05.795 UTC [25731] WARNING: there is no transaction in progress
2026-08-12 03:05:05.811 UTC [25731] WARNING: there is no transaction in progress
2026-08-12 03:05:05.827 UTC [25731] WARNING: there is no transaction in progress
2026-08-12 03:05:05.840 UTC [25731] WARNING: there is no transaction in progress
2026-08-12 03:05:05.853 UTC [25731] WARNING: there is no transaction in progress
2026-08-12 03:05:05.867 UTC [25731] WARNING: there is no transaction in progress
2026-08-12 03:05:05.913 UTC [25731] WARNING: there is no transaction in progress
2026-08-12 03:05:05.919 UTC [25731] WARNING: there is no transaction in progress
2026-08-12 03:05:05.930 UTC [25731] WARNING: there is no transaction in progress
2026-08-12 03:05:05.939 UTC [25731] WARNING: there is no transaction in progress
2026-08-12 03:05:05.947 UTC [25731] WARNING: there is no transaction in progress
2026-08-12 03:05:05.959 UTC [25731] WARNING: there is no transaction in progress
2026-08-12 03:05:05.967 UTC [25731] WARNING: there is no transaction in progress
2026-08-12 03:05:05.972 UTC [25731] WARNING: there is no transaction in progress
2026-08-12 03:05:05.978 UTC [25731] WARNING: there is no transaction in progress
2026-08-12 03:05:05.983 UTC [25731] WARNING: there is no transaction in progress
2026-08-12 03:05:06,470 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 03:05:06,470 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:36171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 03:05:06,714 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 03:05:06,714 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:36171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 03:05:06,961 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 03:05:06,961 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:36171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 03:05:07,211 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 03:05:07,211 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:36171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 03:05:07,449 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:36171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 03:05:07,700 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:36171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6bbx.c.gcp-upstream.internal';"
2026-08-12 03:05:07,938 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 03:05:07,939 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:36171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 03:05:08,173 [I] DB_PATCHES have already been applied
I20260812 03:05:08.178778 5506 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ranger-admin:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task6aqATZ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 03:05:08.178836 5506 mini_ranger.cc:193] Using host: 127.5.96.148
I20260812 03:05:08.182969 5506 mini_ranger.cc:227] Ranger admin URL: http://127.5.96.148:35169
Aug 12, 2026 3:05:08 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 3:05:08 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 3:05:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 3:05:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 3:05:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 3:05:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 3:05:09 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 03:05:09.405921 5506 mini_ranger.cc:160] Time spent starting Ranger: real 7.918s user 0.000s sys 0.010s
Aug 12, 2026 3:05:29 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 3:05:29 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 3:05:29 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 3:05:29 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 3:05:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 3:05:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 3:05:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 3:05:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 3:05:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 3:05:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 3:05:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 3:05:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 3:05:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 3:05:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 3:05:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 3:05:29 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 3:05:29 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 3:05:30 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 03:05:30.553707 5506 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.96.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.96.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.96.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/rangerkms_127.5.96.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.96.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/rangerkms_127.5.96.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.96.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.96.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.96.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/HTTP_127.5.96.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.96.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/HTTP_127.5.96.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 03:05:30 dist-test-slave-6bbx krb5kdc[25319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503930, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 03:05:30.627656 5506 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 03:05:30.636734 5506 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 03:05:31.257612 5506 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 03:05:31.257694 5506 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ranger-kms
I20260812 03:05:31.257714 5506 mini_ranger_kms.cc:227] Using postgres at 127.5.96.148:36171
2026-08-12 03:05:31,416 [I] DB FLAVOR :POSTGRES
2026-08-12 03:05:31,416 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 03:05:31,416 [I] Checking connection
2026-08-12 03:05:31,416 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:36171/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 03:05:31,646 [I] connection success
2026-08-12 03:05:31,646 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 03:05:31,646 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 03:05:31,646 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:36171/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 03:05:31,870 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 03:05:31,870 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 03:05:31,871 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 03:05:31,871 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.96.148:36171/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 03:05:32,106 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 03:05:32.533349 5506 mini_ranger_kms.cc:310] Created kms service
I20260812 03:05:32.734831 5506 mini_ranger_kms.cc:326] Created kudu user
I20260812 03:05:32.781823 5506 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 03:05:32 dist-test-slave-6bbx krb5kdc[25319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503930, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.96.148@KRBTEST.COM
I20260812 03:05:33.158140 5506 mini_ranger_kms.cc:384] Added ranger policy
I20260812 03:05:33.158396 5506 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ranger-kms:/tmp/dist-test-task6aqATZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task6aqATZ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 03:05:33.158427 5506 mini_ranger_kms.cc:240] Using host: 127.5.96.148
I20260812 03:05:33.163331 5506 mini_ranger_kms.cc:276] Ranger KMS PID: 26075
I20260812 03:05:33.163409 5506 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.96.148:33199
Aug 12, 2026 3:05:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 3:05:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 3:05:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 3:05:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 3:05:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 3:05:34 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 03:05:35 dist-test-slave-6bbx krb5kdc[25319](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786503935, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.96.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:05:35.718101 5506 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.090s user 0.002s sys 0.142s
I20260812 03:05:35.718225 5506 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 03:05:35.718286 5506 mini_ranger_kms.cc:401] 127.5.96.148:33199/kms/v1/keys
Aug 12 03:05:35 dist-test-slave-6bbx krb5kdc[25319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503935, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.96.148@KRBTEST.COM for HTTP/127.5.96.148@KRBTEST.COM
Aug 12 03:05:36 dist-test-slave-6bbx krb5kdc[25319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503935, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.96.148@KRBTEST.COM for HTTP/127.5.96.148@KRBTEST.COM
Aug 12 03:05:38 dist-test-slave-6bbx krb5kdc[25319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:05:38.353914 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:40419
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:40419
--ranger_config_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.96.148:33199/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:05:38.459607 26159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:05:38.459904 26159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:05:38.459973 26159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:05:38.463835 26159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:05:38.463882 26159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:05:38.463907 26159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:05:38.463938 26159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:05:38.463966 26159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:05:38.469076 26159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.96.148:33199/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:40419
--ranger_config_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:40419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:05:38.469969 26159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:05:38.470873 26159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:05:38.478673 26174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:38.478695 26171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:38.478806 26159 server_base.cc:1034] running on GCE node
W20260812 03:05:38.478717 26168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:38.479420 26159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:05:38.480443 26159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:05:38.481586 26159 hybrid_clock.cc:648] HybridClock initialized: now 1786503938481569 us; error 29 us; skew 500 ppm
Aug 12 03:05:38 dist-test-slave-6bbx krb5kdc[25319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503938, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:05:38.484194 26159 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:05:38.484972 26159 webserver.cc:460] Webserver started at http://127.5.96.190:36809/ using document root <none> and password file <none>
I20260812 03:05:38.485450 26159 fs_manager.cc:359] Metadata directory not provided
I20260812 03:05:38.485505 26159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:05:38.485684 26159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:05:38 dist-test-slave-6bbx krb5kdc[25319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503938, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for HTTP/127.5.96.148@KRBTEST.COM
I20260812 03:05:38.533689 26159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "7f5f3ff51a5e4ccebeb4a5f3d8101ebf"
format_stamp: "Formatted at 2026-08-12 03:05:38 on dist-test-slave-6bbx"
server_key: "12914ec7a6a0e981483d8a106eb2a52c"
server_key_iv: "ddd696a037ea8f94a9d36c5c12cbc7bd"
server_key_version: "kuduclusterkey@0"
I20260812 03:05:38.534305 26159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "7f5f3ff51a5e4ccebeb4a5f3d8101ebf"
format_stamp: "Formatted at 2026-08-12 03:05:38 on dist-test-slave-6bbx"
server_key: "12914ec7a6a0e981483d8a106eb2a52c"
server_key_iv: "ddd696a037ea8f94a9d36c5c12cbc7bd"
server_key_version: "kuduclusterkey@0"
I20260812 03:05:38.538125 26159 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260812 03:05:38.551848 26190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:38.553144 26159 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:05:38.553274 26159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "7f5f3ff51a5e4ccebeb4a5f3d8101ebf"
format_stamp: "Formatted at 2026-08-12 03:05:38 on dist-test-slave-6bbx"
server_key: "12914ec7a6a0e981483d8a106eb2a52c"
server_key_iv: "ddd696a037ea8f94a9d36c5c12cbc7bd"
server_key_version: "kuduclusterkey@0"
I20260812 03:05:38.553391 26159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:05:38.564536 26159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:05:38.567145 26159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:05:38.567317 26159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:05:38.574519 26159 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:40419
I20260812 03:05:38.574537 26283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:40419 every 8 connection(s)
I20260812 03:05:38.575474 26159 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:05:38.578161 26284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:05:38.579639 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 26159
I20260812 03:05:38.579778 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:05:38.580081 5506 external_mini_cluster.cc:1428] Setting key 38bb64ed8c8ac3ab6217a03a44988f06
I20260812 03:05:38.584712 26284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf: Bootstrap starting.
I20260812 03:05:38.586954 26284 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf: Neither blocks nor log segments found. Creating new log.
I20260812 03:05:38.587757 26284 log.cc:824] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf: Log is configured to *not* fsync() on all Append() calls
I20260812 03:05:38.589900 26284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf: No bootstrap required, opened a new log
Aug 12 03:05:38 dist-test-slave-6bbx krb5kdc[25319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:05:38.593793 26284 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f5f3ff51a5e4ccebeb4a5f3d8101ebf" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40419 } }
I20260812 03:05:38.594134 26284 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:05:38.594427 26284 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f5f3ff51a5e4ccebeb4a5f3d8101ebf, State: Initialized, Role: FOLLOWER
I20260812 03:05:38.594920 26284 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f5f3ff51a5e4ccebeb4a5f3d8101ebf" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40419 } }
I20260812 03:05:38.595078 26284 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:05:38.595183 26284 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:05:38.595235 26290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:38.581492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54662 (local address 127.5.96.190:40419)
0812 03:05:38.581948 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:38.581966 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:38.582009 (+ 43us) server_negotiation.cc:407] Connection header received
0812 03:05:38.582901 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:38.582925 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:38.583323 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:38.583704 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:38.584554 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:38.586116 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:38.587306 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:38.588049 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:38.588202 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:38.590513 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:38.590518 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:38.590530 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:38.590555 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:38.592594 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:38.592995 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:38.593001 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:38.593005 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:38.593106 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:38.593366 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:38.593369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:38.593371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:38.593738 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:38.593925 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:38.594186 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 03:05:38.594455 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":136,"threads_started":1}
I20260812 03:05:38.595288 26284 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:05:38.596309 26284 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f5f3ff51a5e4ccebeb4a5f3d8101ebf" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40419 } }
I20260812 03:05:38.596662 26284 leader_election.cc:302] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f5f3ff51a5e4ccebeb4a5f3d8101ebf; no voters:
I20260812 03:05:38.596961 26284 leader_election.cc:288] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:05:38.597059 26293 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:05:38.597599 26293 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [term 1 LEADER]: Becoming Leader. State: Replica: 7f5f3ff51a5e4ccebeb4a5f3d8101ebf, State: Running, Role: LEADER
I20260812 03:05:38.597888 26293 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f5f3ff51a5e4ccebeb4a5f3d8101ebf" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40419 } }
I20260812 03:05:38.598259 26284 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:05:38.598906 26293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f5f3ff51a5e4ccebeb4a5f3d8101ebf. Latest consensus state: current_term: 1 leader_uuid: "7f5f3ff51a5e4ccebeb4a5f3d8101ebf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f5f3ff51a5e4ccebeb4a5f3d8101ebf" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40419 } } }
I20260812 03:05:38.599102 26293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [sys.catalog]: This master's current role is: LEADER
I20260812 03:05:38.599354 26294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f5f3ff51a5e4ccebeb4a5f3d8101ebf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f5f3ff51a5e4ccebeb4a5f3d8101ebf" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40419 } } }
I20260812 03:05:38.599495 26294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [sys.catalog]: This master's current role is: LEADER
I20260812 03:05:38.599545 26305 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:05:38.602628 26305 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:05:38.603188 26284 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 03:05:38.613022 26305 catalog_manager.cc:1347] Generated new cluster ID: 848f4816eaeb4a0797aef9466a5c684b
I20260812 03:05:38.613081 26305 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:05:38.624136 26305 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:05:38.625288 26305 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:05:38.636663 26305 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf: Generated new TSK 0
I20260812 03:05:38.637434 26305 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.96.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
I20260812 03:05:39.247641 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.96.148:33199/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:0
--local_ip_for_outbound_sockets=127.5.96.129
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:40419
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:05:39.360903 26357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:05:39.361124 26357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:05:39.361179 26357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:05:39.365168 26357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:05:39.365223 26357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:05:39.365304 26357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:05:39.370761 26357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.96.148:33199/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:40419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:05:39.371799 26357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:05:39.372789 26357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:05:39.381829 26366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:39.381834 26365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:39.382040 26357 server_base.cc:1034] running on GCE node
W20260812 03:05:39.382220 26370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:39.382639 26357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:05:39.383397 26357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:05:39.384626 26357 hybrid_clock.cc:648] HybridClock initialized: now 1786503939384588 us; error 51 us; skew 500 ppm
Aug 12 03:05:39 dist-test-slave-6bbx krb5kdc[25319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503939, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:05:39.392347 26357 init.cc:377] Logged in from keytab as kudu/127.5.96.129@KRBTEST.COM (short username kudu)
I20260812 03:05:39.393448 26357 webserver.cc:460] Webserver started at http://127.5.96.129:37841/ using document root <none> and password file <none>
I20260812 03:05:39.394086 26357 fs_manager.cc:359] Metadata directory not provided
I20260812 03:05:39.394157 26357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:05:39.394379 26357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:05:39 dist-test-slave-6bbx krb5kdc[25319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503939, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for HTTP/127.5.96.148@KRBTEST.COM
I20260812 03:05:39.410315 26357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/data/instance:
uuid: "e28d76be67d8499ca9ce55410e6e432f"
format_stamp: "Formatted at 2026-08-12 03:05:39 on dist-test-slave-6bbx"
server_key: "b2f0087b3e467f1eeac05b316afe2494"
server_key_iv: "f36b720acca5935dad920cc6302fa678"
server_key_version: "kuduclusterkey@0"
I20260812 03:05:39.411561 26357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance:
uuid: "e28d76be67d8499ca9ce55410e6e432f"
format_stamp: "Formatted at 2026-08-12 03:05:39 on dist-test-slave-6bbx"
server_key: "b2f0087b3e467f1eeac05b316afe2494"
server_key_iv: "f36b720acca5935dad920cc6302fa678"
server_key_version: "kuduclusterkey@0"
I20260812 03:05:39.416445 26357 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:05:39.432628 26383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:39.434243 26357 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:05:39.434365 26357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "e28d76be67d8499ca9ce55410e6e432f"
format_stamp: "Formatted at 2026-08-12 03:05:39 on dist-test-slave-6bbx"
server_key: "b2f0087b3e467f1eeac05b316afe2494"
server_key_iv: "f36b720acca5935dad920cc6302fa678"
server_key_version: "kuduclusterkey@0"
I20260812 03:05:39.434481 26357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:05:39.461324 26357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:05:39.464906 26357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:05:39.465162 26357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:05:39.465890 26357 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:05:39.467005 26357 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:05:39.467118 26357 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:39.467216 26357 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:05:39.467267 26357 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:39.480782 26357 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:39981
I20260812 03:05:39.481963 26357 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:05:39.483769 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 26357
I20260812 03:05:39.483886 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance
I20260812 03:05:39.484198 5506 external_mini_cluster.cc:1428] Setting key 98da2251146c5534c0ea711b40d40ebe
I20260812 03:05:39.482818 26567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:39981 every 8 connection(s)
Aug 12 03:05:39 dist-test-slave-6bbx krb5kdc[25319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503939, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:05:39.500126 26571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:39.485351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:42559 (local address 127.5.96.190:40419)
0812 03:05:39.485645 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:39.485653 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:39.486460 (+ 807us) server_negotiation.cc:407] Connection header received
0812 03:05:39.487587 (+ 1127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:39.487590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:39.487645 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:39.487734 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:39.489272 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:39.490017 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:39.491612 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:39.492014 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:39.493777 (+ 1763us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:39.494935 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:39.494939 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:39.494942 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:39.494969 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:39.496876 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:39.497523 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:39.497526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:39.497528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:39.497620 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:39.498036 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:39.498039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:39.498040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:39.498268 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:39.499728 (+ 1460us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:39.499811 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 03:05:39.499972 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":118,"threads_started":1}
I20260812 03:05:39.500255 26574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:39.485763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:40419 (local address 127.5.96.129:42559)
0812 03:05:39.486220 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:39.486273 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:39.487356 (+ 1083us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:39.488151 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:39.488160 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:39.488465 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:05:39.489108 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:39.489127 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:39.490191 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:39.490195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:39.491461 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:39.491473 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:39.492196 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:39.492200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:39.492409 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:39.493063 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:05:39.493082 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:05:39.494751 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:05:39.497012 (+ 2261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:39.497018 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:39.497033 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:39.497375 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:39.497731 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:39.497734 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:39.497736 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:39.497917 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:39.498393 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:05:39.498399 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:05:39.498678 (+ 279us) client_negotiation.cc:769] Sending connection context
0812 03:05:39.498860 (+ 182us) client_negotiation.cc:240] Negotiation successful
0812 03:05:39.499197 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":108,"threads_started":1}
WARNING: no policy specified for kudu/127.5.96.130@KRBTEST.COM; defaulting to no policy
I20260812 03:05:39.501786 26568 heartbeater.cc:347] Connected to a master server at 127.5.96.190:40419
I20260812 03:05:39.502015 26568 heartbeater.cc:464] Registering TS with master...
I20260812 03:05:39.502547 26568 heartbeater.cc:511] Master 127.5.96.190:40419 requested a full tablet report, sending...
I20260812 03:05:39.504257 26216 ts_manager.cc:194] Registered new tserver with Master: e28d76be67d8499ca9ce55410e6e432f (127.5.96.129:39981)
I20260812 03:05:39.505752 26216 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.129@KRBTEST.COM'} at 127.5.96.129:42559
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
I20260812 03:05:39.522563 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.96.148:33199/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:0
--local_ip_for_outbound_sockets=127.5.96.130
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:40419
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:05:39 dist-test-slave-6bbx krb5kdc[25319](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786503939, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 03:05:39.680176 26582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:05:39.680501 26582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:05:39.680564 26582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:05:39.686502 26582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:05:39.686604 26582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:05:39.686704 26582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:05:39.694289 26582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.96.148:33199/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:40419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:05:39.695559 26582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:05:39.696926 26582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:05:39.708838 26602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:39.709628 26590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:39.709779 26582 server_base.cc:1034] running on GCE node
W20260812 03:05:39.709894 26592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:39.710491 26582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:05:39.711205 26582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:05:39.712391 26582 hybrid_clock.cc:648] HybridClock initialized: now 1786503939712364 us; error 41 us; skew 500 ppm
Aug 12 03:05:39 dist-test-slave-6bbx krb5kdc[25319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503939, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:05:39.715996 26582 init.cc:377] Logged in from keytab as kudu/127.5.96.130@KRBTEST.COM (short username kudu)
I20260812 03:05:39.717022 26582 webserver.cc:460] Webserver started at http://127.5.96.130:44529/ using document root <none> and password file <none>
I20260812 03:05:39.717718 26582 fs_manager.cc:359] Metadata directory not provided
I20260812 03:05:39.717851 26582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:05:39.718156 26582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:05:39 dist-test-slave-6bbx krb5kdc[25319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503939, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for HTTP/127.5.96.148@KRBTEST.COM
I20260812 03:05:39.733716 26582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/data/instance:
uuid: "de300ba4028849a48302567a03a85cf3"
format_stamp: "Formatted at 2026-08-12 03:05:39 on dist-test-slave-6bbx"
server_key: "f56f7248d12164721196df6acf86cea1"
server_key_iv: "5c40f08bbc98cdd29f45a13472dfd82a"
server_key_version: "kuduclusterkey@0"
I20260812 03:05:39.734519 26582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance:
uuid: "de300ba4028849a48302567a03a85cf3"
format_stamp: "Formatted at 2026-08-12 03:05:39 on dist-test-slave-6bbx"
server_key: "f56f7248d12164721196df6acf86cea1"
server_key_iv: "5c40f08bbc98cdd29f45a13472dfd82a"
server_key_version: "kuduclusterkey@0"
I20260812 03:05:39.742668 26582 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.000s sys 0.004s
I20260812 03:05:39.757840 26613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:39.759441 26582 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:05:39.759558 26582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "de300ba4028849a48302567a03a85cf3"
format_stamp: "Formatted at 2026-08-12 03:05:39 on dist-test-slave-6bbx"
server_key: "f56f7248d12164721196df6acf86cea1"
server_key_iv: "5c40f08bbc98cdd29f45a13472dfd82a"
server_key_version: "kuduclusterkey@0"
I20260812 03:05:39.759687 26582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:05:39.780817 26582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:05:39.783787 26582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:05:39.784005 26582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:05:39.784693 26582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:05:39.785849 26582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:05:39.785949 26582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:39.786046 26582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:05:39.786121 26582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:39.797832 26582 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:39639
I20260812 03:05:39.798962 26582 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:05:39.802937 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 26582
I20260812 03:05:39.803057 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance
I20260812 03:05:39.799616 26823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:39639 every 8 connection(s)
I20260812 03:05:39.803416 5506 external_mini_cluster.cc:1428] Setting key df455862fb0b4e583bbcf540e5ace48b
Aug 12 03:05:39 dist-test-slave-6bbx krb5kdc[25319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503939, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.96.131@KRBTEST.COM; defaulting to no policy
I20260812 03:05:39.819948 26571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:39.802853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:36843 (local address 127.5.96.190:40419)
0812 03:05:39.802972 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:39.802976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:39.804294 (+ 1318us) server_negotiation.cc:407] Connection header received
0812 03:05:39.808844 (+ 4550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:39.808848 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:39.808900 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:39.808950 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:39.810280 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:39.811051 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:39.812627 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:39.812945 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:39.813019 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:39.816018 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:39.816021 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:39.816024 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:39.816051 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:39.817588 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:39.818158 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:39.818161 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:39.818163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:39.818241 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:39.818598 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:39.818601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:39.818602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:39.818805 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:39.818886 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:39.819618 (+ 732us) server_negotiation.cc:299] Negotiation successful
0812 03:05:39.819779 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.131@KRBTEST.COM" created.
I20260812 03:05:39.821014 26828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:39.801757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:40419 (local address 127.5.96.130:36843)
0812 03:05:39.803932 (+ 2175us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:39.803988 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:39.808595 (+ 4607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:39.809365 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:39.809375 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:39.809653 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:05:39.810124 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:39.810138 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:39.811205 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:39.811208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:39.812478 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:39.812490 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:39.813216 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:39.813219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:39.813418 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:39.814103 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:05:39.814126 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:05:39.815831 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:05:39.817723 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:39.817728 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:39.817743 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:39.818051 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:39.818350 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:39.818352 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:39.818354 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:39.818506 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:39.818954 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:05:39.818959 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:05:39.819259 (+ 300us) client_negotiation.cc:769] Sending connection context
0812 03:05:39.819528 (+ 269us) client_negotiation.cc:240] Negotiation successful
0812 03:05:39.819868 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1960,"thread_start_us":135,"threads_started":1}
I20260812 03:05:39.822573 26825 heartbeater.cc:347] Connected to a master server at 127.5.96.190:40419
I20260812 03:05:39.822818 26825 heartbeater.cc:464] Registering TS with master...
I20260812 03:05:39.823396 26825 heartbeater.cc:511] Master 127.5.96.190:40419 requested a full tablet report, sending...
I20260812 03:05:39.826265 26216 ts_manager.cc:194] Registered new tserver with Master: de300ba4028849a48302567a03a85cf3 (127.5.96.130:39639)
I20260812 03:05:39.827106 26216 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.130@KRBTEST.COM'} at 127.5.96.130:36843
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
I20260812 03:05:39.855171 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.96.148:33199/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:0
--local_ip_for_outbound_sockets=127.5.96.131
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:40419
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:05:40 dist-test-slave-6bbx krb5kdc[25319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503939, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for HTTP/127.5.96.148@KRBTEST.COM
W20260812 03:05:40.028126 26833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:05:40.028537 26833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:05:40.028661 26833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:05:40.034451 26833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:05:40.034576 26833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:05:40.034706 26833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:05:40.041911 26833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.96.148:33199/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:40419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:05:40.043503 26833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:05:40.044910 26833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:05:40.057888 26845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:40.058326 26847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:40.057898 26844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:40.059190 26833 server_base.cc:1034] running on GCE node
I20260812 03:05:40.059614 26833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:05:40.060433 26833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:05:40.067809 26833 hybrid_clock.cc:648] HybridClock initialized: now 1786503940067778 us; error 39 us; skew 500 ppm
Aug 12 03:05:40 dist-test-slave-6bbx krb5kdc[25319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503940, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:05:40.071283 26833 init.cc:377] Logged in from keytab as kudu/127.5.96.131@KRBTEST.COM (short username kudu)
I20260812 03:05:40.072418 26833 webserver.cc:460] Webserver started at http://127.5.96.131:36339/ using document root <none> and password file <none>
I20260812 03:05:40.073115 26833 fs_manager.cc:359] Metadata directory not provided
I20260812 03:05:40.073204 26833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:05:40.073436 26833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:05:40 dist-test-slave-6bbx krb5kdc[25319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503940, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for HTTP/127.5.96.148@KRBTEST.COM
I20260812 03:05:40.089773 26833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/data/instance:
uuid: "776402ce26e947669a65ccd4d40e97c4"
format_stamp: "Formatted at 2026-08-12 03:05:40 on dist-test-slave-6bbx"
server_key: "567cfe0c83a9dc5d6d3549a551b6b4d3"
server_key_iv: "754cfc6a494519dd92c17d0d14f45c4c"
server_key_version: "kuduclusterkey@0"
I20260812 03:05:40.090436 26833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance:
uuid: "776402ce26e947669a65ccd4d40e97c4"
format_stamp: "Formatted at 2026-08-12 03:05:40 on dist-test-slave-6bbx"
server_key: "567cfe0c83a9dc5d6d3549a551b6b4d3"
server_key_iv: "754cfc6a494519dd92c17d0d14f45c4c"
server_key_version: "kuduclusterkey@0"
I20260812 03:05:40.095803 26833 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 03:05:40.120067 26858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:40.121673 26833 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.003s sys 0.000s
I20260812 03:05:40.121804 26833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "776402ce26e947669a65ccd4d40e97c4"
format_stamp: "Formatted at 2026-08-12 03:05:40 on dist-test-slave-6bbx"
server_key: "567cfe0c83a9dc5d6d3549a551b6b4d3"
server_key_iv: "754cfc6a494519dd92c17d0d14f45c4c"
server_key_version: "kuduclusterkey@0"
I20260812 03:05:40.121934 26833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:05:40.141106 26833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:05:40.144572 26833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:05:40.144815 26833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:05:40.145494 26833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:05:40.146589 26833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:05:40.146713 26833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:40.146821 26833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:05:40.146895 26833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:40.157333 26833 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:44157
I20260812 03:05:40.158586 26833 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:05:40.160801 27063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:44157 every 8 connection(s)
I20260812 03:05:40.163816 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 26833
I20260812 03:05:40.163946 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance
I20260812 03:05:40.164261 5506 external_mini_cluster.cc:1428] Setting key 7c56d426a983f677471f638f7b9c9ef9
Aug 12 03:05:40 dist-test-slave-6bbx krb5kdc[25319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503940, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:05:40.178179 26571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:40.162777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:38751 (local address 127.5.96.190:40419)
0812 03:05:40.162915 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:40.162920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:40.164873 (+ 1953us) server_negotiation.cc:407] Connection header received
0812 03:05:40.166248 (+ 1375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:40.166252 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:40.166307 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:40.166382 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:40.167954 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:40.168719 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:40.170379 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:40.170700 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:40.170820 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:40.173560 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:40.173564 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:40.173567 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:40.173607 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:40.175209 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:40.175799 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:40.175803 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:40.175804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:40.175902 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:40.176256 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:40.176259 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:40.176261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:40.176483 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:40.176588 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:40.177880 (+ 1292us) server_negotiation.cc:299] Negotiation successful
0812 03:05:40.178016 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 03:05:40.179315 27067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:40.163298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:40419 (local address 127.5.96.131:38751)
0812 03:05:40.164706 (+ 1408us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:40.164764 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:40.165883 (+ 1119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:40.166707 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:40.166717 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:40.167067 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:05:40.167793 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:40.167815 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:40.168863 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:40.168867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:40.170249 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:40.170261 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:40.170885 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:40.170891 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:40.171097 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:40.171759 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:05:40.171779 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:05:40.173381 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:05:40.175329 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:40.175335 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:40.175350 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:40.175694 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:40.175989 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:40.175991 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:40.175993 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:40.176156 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:40.176588 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:05:40.176595 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:05:40.177212 (+ 617us) client_negotiation.cc:769] Sending connection context
0812 03:05:40.177577 (+ 365us) client_negotiation.cc:240] Negotiation successful
0812 03:05:40.178347 (+ 770us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1082,"thread_start_us":152,"threads_started":1}
I20260812 03:05:40.180413 27064 heartbeater.cc:347] Connected to a master server at 127.5.96.190:40419
I20260812 03:05:40.180639 27064 heartbeater.cc:464] Registering TS with master...
I20260812 03:05:40.181178 27064 heartbeater.cc:511] Master 127.5.96.190:40419 requested a full tablet report, sending...
I20260812 03:05:40.182264 26216 ts_manager.cc:194] Registered new tserver with Master: 776402ce26e947669a65ccd4d40e97c4 (127.5.96.131:44157)
I20260812 03:05:40.183002 26216 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.131@KRBTEST.COM'} at 127.5.96.131:38751
I20260812 03:05:40.189805 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:05:40.190495 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 26357
I20260812 03:05:40.208173 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 26582
I20260812 03:05:40.239789 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 26833
I20260812 03:05:40.249910 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 26159
I20260812 03:05:40.266117 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:40419
--webserver_interface=127.5.96.190
--webserver_port=36809
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:40419
--ranger_config_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.96.148:33199/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.96.148:33199/kms with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:05:40.372422 27068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:05:40.372635 27068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:05:40.372671 27068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:05:40.376492 27068 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:05:40.376536 27068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:05:40.376554 27068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:05:40.376573 27068 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:05:40.376587 27068 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:05:40.381642 27068 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.96.148:33199/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:40419
--ranger_config_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:40419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=36809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:05:40.382475 27068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:05:40.383328 27068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:05:40.391198 27076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:40.391189 27075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:40.391234 27068 server_base.cc:1034] running on GCE node
W20260812 03:05:40.391187 27078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:40.391957 27068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:05:40.392931 27068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:05:40.394084 27068 hybrid_clock.cc:648] HybridClock initialized: now 1786503940394072 us; error 29 us; skew 500 ppm
Aug 12 03:05:40 dist-test-slave-6bbx krb5kdc[25319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503940, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:05:40.396586 27068 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:05:40.397357 27068 webserver.cc:460] Webserver started at http://127.5.96.190:36809/ using document root <none> and password file <none>
I20260812 03:05:40.397830 27068 fs_manager.cc:359] Metadata directory not provided
I20260812 03:05:40.397902 27068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 03:05:40 dist-test-slave-6bbx krb5kdc[25319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503940, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for HTTP/127.5.96.148@KRBTEST.COM
I20260812 03:05:40.432894 27068 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:05:40.435092 27090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:40.436431 27068 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:05:40.436548 27068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "7f5f3ff51a5e4ccebeb4a5f3d8101ebf"
format_stamp: "Formatted at 2026-08-12 03:05:38 on dist-test-slave-6bbx"
server_key: "12914ec7a6a0e981483d8a106eb2a52c"
server_key_iv: "ddd696a037ea8f94a9d36c5c12cbc7bd"
server_key_version: "kuduclusterkey@0"
I20260812 03:05:40.437079 27068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:05:40.448019 27068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:05:40.450518 27068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:05:40.450683 27068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:05:40.457573 27068 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:40419
I20260812 03:05:40.457592 27206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:40419 every 8 connection(s)
I20260812 03:05:40.458575 27068 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:05:40.462589 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 27068
I20260812 03:05:40.462857 27207 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:05:40.463006 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.96.148:33199/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:39981
--local_ip_for_outbound_sockets=127.5.96.129
--tserver_master_addrs=127.5.96.190:40419
--webserver_port=37841
--webserver_interface=127.5.96.129
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.96.148:33199/kms with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:05:40.464545 27207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf: Bootstrap starting.
I20260812 03:05:40.480379 27207 log.cc:824] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf: Log is configured to *not* fsync() on all Append() calls
I20260812 03:05:40.486681 27207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:05:40.487007 27207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf: Bootstrap complete.
I20260812 03:05:40.490870 27207 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f5f3ff51a5e4ccebeb4a5f3d8101ebf" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40419 } }
I20260812 03:05:40.491242 27207 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f5f3ff51a5e4ccebeb4a5f3d8101ebf, State: Initialized, Role: FOLLOWER
I20260812 03:05:40.491776 27207 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f5f3ff51a5e4ccebeb4a5f3d8101ebf" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40419 } }
I20260812 03:05:40.491911 27207 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:05:40.491979 27207 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:05:40.492074 27207 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:05:40.494021 27207 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f5f3ff51a5e4ccebeb4a5f3d8101ebf" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40419 } }
I20260812 03:05:40.494328 27207 leader_election.cc:302] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f5f3ff51a5e4ccebeb4a5f3d8101ebf; no voters:
I20260812 03:05:40.494606 27207 leader_election.cc:288] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 03:05:40.494725 27211 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:05:40.494953 27211 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [term 2 LEADER]: Becoming Leader. State: Replica: 7f5f3ff51a5e4ccebeb4a5f3d8101ebf, State: Running, Role: LEADER
I20260812 03:05:40.495332 27211 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f5f3ff51a5e4ccebeb4a5f3d8101ebf" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40419 } }
I20260812 03:05:40.495800 27207 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:05:40.496762 27213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7f5f3ff51a5e4ccebeb4a5f3d8101ebf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f5f3ff51a5e4ccebeb4a5f3d8101ebf" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40419 } } }
I20260812 03:05:40.496888 27213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [sys.catalog]: This master's current role is: LEADER
I20260812 03:05:40.497157 27212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f5f3ff51a5e4ccebeb4a5f3d8101ebf. Latest consensus state: current_term: 2 leader_uuid: "7f5f3ff51a5e4ccebeb4a5f3d8101ebf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f5f3ff51a5e4ccebeb4a5f3d8101ebf" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40419 } } }
I20260812 03:05:40.497314 27212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf [sys.catalog]: This master's current role is: LEADER
I20260812 03:05:40.497615 27224 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:05:40.497961 27207 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 03:05:40.503037 27224 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:05:40.503767 27224 catalog_manager.cc:1259] Loaded cluster ID: 848f4816eaeb4a0797aef9466a5c684b
I20260812 03:05:40.503862 27224 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:05:40.506438 27224 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:05:40.507443 27224 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf: Loaded TSK: 0
I20260812 03:05:40.509086 27224 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:05:40.586375 27209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:05:40.586712 27209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:05:40.586782 27209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:05:40.592720 27209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:05:40.592785 27209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:05:40.592864 27209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:05:40.600564 27209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.96.148:33199/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:39981
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=37841
--tserver_master_addrs=127.5.96.190:40419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:05:40.601859 27209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:05:40.603114 27209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:05:40.616153 27257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:40.616487 27209 server_base.cc:1034] running on GCE node
W20260812 03:05:40.616551 27264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:40.619802 27258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:40.620275 27209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:05:40.621043 27209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:05:40.622223 27209 hybrid_clock.cc:648] HybridClock initialized: now 1786503940622194 us; error 50 us; skew 500 ppm
Aug 12 03:05:40 dist-test-slave-6bbx krb5kdc[25319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503940, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:05:40.625866 27209 init.cc:377] Logged in from keytab as kudu/127.5.96.129@KRBTEST.COM (short username kudu)
I20260812 03:05:40.627154 27209 webserver.cc:460] Webserver started at http://127.5.96.129:37841/ using document root <none> and password file <none>
I20260812 03:05:40.627907 27209 fs_manager.cc:359] Metadata directory not provided
I20260812 03:05:40.628042 27209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 03:05:40 dist-test-slave-6bbx krb5kdc[25319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503940, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for HTTP/127.5.96.148@KRBTEST.COM
I20260812 03:05:40.659523 27209 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.003s sys 0.000s
I20260812 03:05:40.662814 27278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:40.664415 27209 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 03:05:40.664528 27209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "e28d76be67d8499ca9ce55410e6e432f"
format_stamp: "Formatted at 2026-08-12 03:05:39 on dist-test-slave-6bbx"
server_key: "b2f0087b3e467f1eeac05b316afe2494"
server_key_iv: "f36b720acca5935dad920cc6302fa678"
server_key_version: "kuduclusterkey@0"
I20260812 03:05:40.665225 27209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:05:40.688874 27209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:05:40.692134 27209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:05:40.692307 27209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:05:40.692924 27209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:05:40.693848 27209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:05:40.693902 27209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:40.693953 27209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:05:40.693996 27209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:40.705720 27209 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:39981
I20260812 03:05:40.705773 27500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:39981 every 8 connection(s)
I20260812 03:05:40.706892 27209 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:05:40.713776 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 27209
I20260812 03:05:40.714252 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.96.148:33199/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:39639
--local_ip_for_outbound_sockets=127.5.96.130
--tserver_master_addrs=127.5.96.190:40419
--webserver_port=44529
--webserver_interface=127.5.96.130
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.96.148:33199/kms with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:05:40 dist-test-slave-6bbx krb5kdc[25319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503940, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:05:40.727238 27507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:40.709262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:40419 (local address 127.5.96.129:43319)
0812 03:05:40.710162 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:40.710207 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:40.711180 (+ 973us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:40.713291 (+ 2111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:40.713300 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:40.713647 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:05:40.714275 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:40.714295 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:40.716004 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:40.716008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:40.717379 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:40.717391 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:40.718594 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:40.718598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:40.718787 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:40.719365 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:05:40.719385 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:05:40.721127 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:05:40.723594 (+ 2467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:40.723600 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:40.723613 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:40.723983 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:40.724456 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:40.724460 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:40.724462 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:40.724650 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:40.725461 (+ 811us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:05:40.725467 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:05:40.725748 (+ 281us) client_negotiation.cc:769] Sending connection context
0812 03:05:40.725942 (+ 194us) client_negotiation.cc:240] Negotiation successful
0812 03:05:40.726255 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"thread_start_us":124,"threads_started":1}
I20260812 03:05:40.728950 27506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:40.709099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:43319 (local address 127.5.96.190:40419)
0812 03:05:40.711691 (+ 2592us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:40.711714 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:40.711757 (+ 43us) server_negotiation.cc:407] Connection header received
0812 03:05:40.712050 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:40.712065 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:40.712475 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:40.712808 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:40.714475 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:40.715859 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:40.717532 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:40.718234 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:40.718388 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:40.721333 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:40.721339 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:40.721353 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:40.721382 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:40.723439 (+ 2057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:40.724134 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:40.724140 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:40.724146 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:40.724309 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:40.724772 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:40.724776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:40.724778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:40.725262 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:40.725399 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:40.727749 (+ 2350us) server_negotiation.cc:299] Negotiation successful
0812 03:05:40.728025 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2411,"thread_start_us":111,"threads_started":1}
I20260812 03:05:40.731523 27501 heartbeater.cc:347] Connected to a master server at 127.5.96.190:40419
I20260812 03:05:40.731782 27501 heartbeater.cc:464] Registering TS with master...
I20260812 03:05:40.732326 27501 heartbeater.cc:511] Master 127.5.96.190:40419 requested a full tablet report, sending...
W20260812 03:05:40.734414 27501 heartbeater.cc:644] Failed to heartbeat to 127.5.96.190:40419 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 03:05:40.734526 27501 heartbeater.cc:464] Registering TS with master...
I20260812 03:05:40.734684 27501 heartbeater.cc:511] Master 127.5.96.190:40419 requested a full tablet report, sending...
I20260812 03:05:40.735543 27501 heartbeater.cc:464] Registering TS with master...
I20260812 03:05:40.735765 27501 heartbeater.cc:511] Master 127.5.96.190:40419 requested a full tablet report, sending...
W20260812 03:05:40.736687 27501 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:05:40.861795 27509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:05:40.862023 27509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:05:40.862078 27509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:05:40.866134 27509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:05:40.866190 27509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:05:40.866242 27509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:05:40.871701 27509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.96.148:33199/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:39639
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=44529
--tserver_master_addrs=127.5.96.190:40419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:05:40.872632 27509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:05:40.873570 27509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:05:40.882918 27519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:40.883095 27526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:40.883281 27521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:40.883332 27509 server_base.cc:1034] running on GCE node
I20260812 03:05:40.883882 27509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:05:40.884632 27509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:05:40.885840 27509 hybrid_clock.cc:648] HybridClock initialized: now 1786503940885811 us; error 39 us; skew 500 ppm
Aug 12 03:05:40 dist-test-slave-6bbx krb5kdc[25319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503940, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:05:40.889401 27509 init.cc:377] Logged in from keytab as kudu/127.5.96.130@KRBTEST.COM (short username kudu)
I20260812 03:05:40.890503 27509 webserver.cc:460] Webserver started at http://127.5.96.130:44529/ using document root <none> and password file <none>
I20260812 03:05:40.891192 27509 fs_manager.cc:359] Metadata directory not provided
I20260812 03:05:40.891314 27509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 03:05:40 dist-test-slave-6bbx krb5kdc[25319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503940, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for HTTP/127.5.96.148@KRBTEST.COM
I20260812 03:05:40.910871 27509 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.003s
I20260812 03:05:40.913754 27540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:40.915159 27509 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:05:40.915431 27509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "de300ba4028849a48302567a03a85cf3"
format_stamp: "Formatted at 2026-08-12 03:05:39 on dist-test-slave-6bbx"
server_key: "f56f7248d12164721196df6acf86cea1"
server_key_iv: "5c40f08bbc98cdd29f45a13472dfd82a"
server_key_version: "kuduclusterkey@0"
I20260812 03:05:40.916146 27509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:05:40.938181 27509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:05:40.941632 27509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:05:40.941869 27509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:05:40.942646 27509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:05:40.943718 27509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:05:40.943819 27509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:40.943918 27509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:05:40.943984 27509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:40.955550 27509 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:39639
I20260812 03:05:40.955596 27741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:39639 every 8 connection(s)
I20260812 03:05:40.957391 27509 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:05:40.958307 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 27509
I20260812 03:05:40.958819 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.96.148:33199/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:44157
--local_ip_for_outbound_sockets=127.5.96.131
--tserver_master_addrs=127.5.96.190:40419
--webserver_port=36339
--webserver_interface=127.5.96.131
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.96.148:33199/kms with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:05:40 dist-test-slave-6bbx krb5kdc[25319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503940, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:05:40.977743 27506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:40.963542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:34643 (local address 127.5.96.190:40419)
0812 03:05:40.963700 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:40.963704 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:40.965003 (+ 1299us) server_negotiation.cc:407] Connection header received
0812 03:05:40.966234 (+ 1231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:40.966238 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:40.966297 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:40.966356 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:40.967984 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:40.968818 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:40.970492 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:40.970846 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:40.970930 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:40.973635 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:40.973640 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:40.973642 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:40.973670 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:40.975284 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:40.975944 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:40.975947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:40.975949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:40.976038 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:40.976449 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:40.976453 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:40.976455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:40.976676 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:40.976780 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:40.977438 (+ 658us) server_negotiation.cc:299] Negotiation successful
0812 03:05:40.977565 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 03:05:40.979427 27748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:40.962525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:40419 (local address 127.5.96.130:34643)
0812 03:05:40.964843 (+ 2318us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:40.964900 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:40.965978 (+ 1078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:40.966784 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:40.966793 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:40.967114 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:05:40.967802 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:40.967823 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:40.968988 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:40.968992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:40.970343 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:40.970355 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:40.971136 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:40.971141 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:40.971337 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:40.971982 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:05:40.972002 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:05:40.973444 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:05:40.975414 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:40.975420 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:40.975433 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:40.975825 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:40.976143 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:40.976146 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:40.976148 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:40.976330 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:40.976856 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:05:40.976862 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:05:40.977150 (+ 288us) client_negotiation.cc:769] Sending connection context
0812 03:05:40.978140 (+ 990us) client_negotiation.cc:240] Negotiation successful
0812 03:05:40.978453 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2077,"thread_start_us":140,"threads_started":1}
I20260812 03:05:40.980895 27742 heartbeater.cc:347] Connected to a master server at 127.5.96.190:40419
I20260812 03:05:40.981115 27742 heartbeater.cc:464] Registering TS with master...
I20260812 03:05:40.981657 27742 heartbeater.cc:511] Master 127.5.96.190:40419 requested a full tablet report, sending...
W20260812 03:05:40.983377 27742 heartbeater.cc:644] Failed to heartbeat to 127.5.96.190:40419 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 03:05:40.983562 27742 heartbeater.cc:464] Registering TS with master...
I20260812 03:05:40.983837 27742 heartbeater.cc:511] Master 127.5.96.190:40419 requested a full tablet report, sending...
I20260812 03:05:40.984714 27742 heartbeater.cc:464] Registering TS with master...
I20260812 03:05:40.984894 27742 heartbeater.cc:511] Master 127.5.96.190:40419 requested a full tablet report, sending...
W20260812 03:05:40.985710 27742 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:05:41.097837 27747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:05:41.098152 27747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:05:41.098217 27747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:05:41.102695 27747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:05:41.102754 27747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:05:41.102828 27747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:05:41.108031 27747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.96.148:33199/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:44157
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=36339
--tserver_master_addrs=127.5.96.190:40419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:05:41.108946 27747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:05:41.110162 27747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:05:41.119998 27759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:41.119998 27760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:41.120373 27766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:41.120795 27747 server_base.cc:1034] running on GCE node
I20260812 03:05:41.121224 27747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:05:41.121995 27747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:05:41.123229 27747 hybrid_clock.cc:648] HybridClock initialized: now 1786503941123199 us; error 45 us; skew 500 ppm
Aug 12 03:05:41 dist-test-slave-6bbx krb5kdc[25319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503941, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:05:41.126125 27747 init.cc:377] Logged in from keytab as kudu/127.5.96.131@KRBTEST.COM (short username kudu)
I20260812 03:05:41.126938 27747 webserver.cc:460] Webserver started at http://127.5.96.131:36339/ using document root <none> and password file <none>
I20260812 03:05:41.127561 27747 fs_manager.cc:359] Metadata directory not provided
I20260812 03:05:41.127735 27747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 03:05:41 dist-test-slave-6bbx krb5kdc[25319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503941, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for HTTP/127.5.96.148@KRBTEST.COM
I20260812 03:05:41.152707 27747 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.002s sys 0.001s
I20260812 03:05:41.155030 27778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:41.156723 27747 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 03:05:41.156858 27747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "776402ce26e947669a65ccd4d40e97c4"
format_stamp: "Formatted at 2026-08-12 03:05:40 on dist-test-slave-6bbx"
server_key: "567cfe0c83a9dc5d6d3549a551b6b4d3"
server_key_iv: "754cfc6a494519dd92c17d0d14f45c4c"
server_key_version: "kuduclusterkey@0"
I20260812 03:05:41.157444 27747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:05:41.178416 27747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:05:41.181481 27747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:05:41.181648 27747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:05:41.182245 27747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:05:41.183092 27747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:05:41.183149 27747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:41.183187 27747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:05:41.183212 27747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:41.195492 27747 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:44157
I20260812 03:05:41.195761 27977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:44157 every 8 connection(s)
I20260812 03:05:41.196768 27747 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:05:41 dist-test-slave-6bbx krb5kdc[25319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503941, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:05:41.207046 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 27747
I20260812 03:05:41.211570 27506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:41.199438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:41507 (local address 127.5.96.190:40419)
0812 03:05:41.199557 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:41.199562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:41.200480 (+ 918us) server_negotiation.cc:407] Connection header received
0812 03:05:41.201485 (+ 1005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:41.201489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:41.201548 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:41.201667 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:41.202767 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:41.203320 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:41.204592 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:41.204821 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:41.204911 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:41.207419 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:41.207423 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:41.207426 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:41.207456 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:41.209165 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:41.209811 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:41.209814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:41.209816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:41.209910 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:41.210318 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:41.210321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:41.210323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:41.210542 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:41.210650 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:41.211286 (+ 636us) server_negotiation.cc:299] Negotiation successful
0812 03:05:41.211407 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:05:41.212746 27980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:41.199881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:40419 (local address 127.5.96.131:41507)
0812 03:05:41.200333 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:41.200383 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:41.201262 (+ 879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:41.201813 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:41.201822 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:41.202112 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:05:41.202609 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:41.202629 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:41.203440 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:41.203444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:41.204477 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:41.204487 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:41.204960 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:41.204964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:41.205151 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:41.205662 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:05:41.205682 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:05:41.207242 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:05:41.209304 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:41.209310 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:41.209325 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:41.209685 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:41.210020 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:41.210024 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:41.210026 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:41.210202 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:41.210706 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:05:41.210712 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:05:41.210999 (+ 287us) client_negotiation.cc:769] Sending connection context
0812 03:05:41.211209 (+ 210us) client_negotiation.cc:240] Negotiation successful
0812 03:05:41.211511 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":118,"threads_started":1}
I20260812 03:05:41.214206 27978 heartbeater.cc:347] Connected to a master server at 127.5.96.190:40419
I20260812 03:05:41.214434 27978 heartbeater.cc:464] Registering TS with master...
I20260812 03:05:41.214996 27978 heartbeater.cc:511] Master 127.5.96.190:40419 requested a full tablet report, sending...
W20260812 03:05:41.216717 27978 heartbeater.cc:644] Failed to heartbeat to 127.5.96.190:40419 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 03:05:41.216840 27978 heartbeater.cc:464] Registering TS with master...
I20260812 03:05:41.217006 27978 heartbeater.cc:511] Master 127.5.96.190:40419 requested a full tablet report, sending...
I20260812 03:05:41.217885 27978 heartbeater.cc:464] Registering TS with master...
I20260812 03:05:41.218051 27978 heartbeater.cc:511] Master 127.5.96.190:40419 requested a full tablet report, sending...
I20260812 03:05:41.218163 27981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:41.207840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54664 (local address 127.5.96.190:40419)
0812 03:05:41.208366 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:41.208370 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:41.208772 (+ 402us) server_negotiation.cc:407] Connection header received
0812 03:05:41.208810 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:41.208812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:41.208857 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:41.208932 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:41.209896 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:41.210686 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:41.211857 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:41.212461 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:41.212562 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:41.214214 (+ 1652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:41.214217 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:41.214220 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:41.214245 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:41.215939 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:41.216422 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:41.216426 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:41.216428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:41.216513 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:41.216820 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:41.216823 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:41.216825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:41.217026 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:41.217221 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:41.217886 (+ 665us) server_negotiation.cc:299] Negotiation successful
0812 03:05:41.218001 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":82,"threads_started":1}
W20260812 03:05:41.219029 27978 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 03:05:41 dist-test-slave-6bbx krb5kdc[25319](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786503941, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:05:41.736908 27501 heartbeater.cc:464] Registering TS with master...
I20260812 03:05:41.737195 27501 heartbeater.cc:511] Master 127.5.96.190:40419 requested a full tablet report, sending...
I20260812 03:05:41.738372 27123 ts_manager.cc:194] Registered new tserver with Master: e28d76be67d8499ca9ce55410e6e432f (127.5.96.129:39981)
I20260812 03:05:41.740175 27123 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.129@KRBTEST.COM'} at 127.5.96.129:43319
Aug 12 03:05:41 dist-test-slave-6bbx krb5kdc[25319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503941, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for HTTP/127.5.96.148@KRBTEST.COM
I20260812 03:05:41.985939 27742 heartbeater.cc:464] Registering TS with master...
I20260812 03:05:41.986233 27742 heartbeater.cc:511] Master 127.5.96.190:40419 requested a full tablet report, sending...
I20260812 03:05:41.987010 27123 ts_manager.cc:194] Registered new tserver with Master: de300ba4028849a48302567a03a85cf3 (127.5.96.130:39639)
I20260812 03:05:41.987844 27123 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.130@KRBTEST.COM'} at 127.5.96.130:34643
I20260812 03:05:42.219230 27978 heartbeater.cc:464] Registering TS with master...
I20260812 03:05:42.219481 27978 heartbeater.cc:511] Master 127.5.96.190:40419 requested a full tablet report, sending...
I20260812 03:05:42.220161 27123 ts_manager.cc:194] Registered new tserver with Master: 776402ce26e947669a65ccd4d40e97c4 (127.5.96.131:44157)
I20260812 03:05:42.220890 27123 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.131@KRBTEST.COM'} at 127.5.96.131:41507
I20260812 03:05:42.226612 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 03:05:42 dist-test-slave-6bbx krb5kdc[25319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503941, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for HTTP/127.5.96.148@KRBTEST.COM
I20260812 03:05:42.244374 28012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:42.228879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54670 (local address 127.5.96.190:40419)
0812 03:05:42.230245 (+ 1366us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:42.230249 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:42.230263 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:05:42.230300 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:42.230302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:42.230345 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:42.233158 (+ 2813us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:42.233680 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:42.234500 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:42.235646 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:42.236007 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:42.236477 (+ 470us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:42.239795 (+ 3318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:42.239800 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:42.239803 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:42.239829 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:42.242060 (+ 2231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:42.242584 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:42.242588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:42.242590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:42.242680 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:42.243001 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:42.243004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:42.243006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:42.243207 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:42.244014 (+ 807us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:42.244095 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 03:05:42.244208 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1293,"thread_start_us":121,"threads_started":1}
I20260812 03:05:42.248471 27123 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54670:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:05:42.253249 27123 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:05:42.281126 28020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:42.273087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:39981 (local address 127.0.0.1:36400)
0812 03:05:42.273560 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:42.273607 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:42.273741 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:42.274562 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:42.274569 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:42.274596 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:05:42.274773 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:42.274782 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:42.276490 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:42.276494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:42.277804 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:42.277817 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:42.280585 (+ 2768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:42.280589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:42.280698 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:42.280751 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 03:05:42.280849 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 03:05:42.280969 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":104,"threads_started":1}
I20260812 03:05:42.281843 28023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:42.273789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36400 (local address 127.5.96.129:39981)
0812 03:05:42.274068 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:42.274074 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:42.274098 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:05:42.274177 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:42.274182 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:42.274385 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:42.275063 (+ 678us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:05:42.275210 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:42.276387 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:42.279099 (+ 2712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:42.280469 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:42.280565 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:42.280926 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:42.280979 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 03:05:42.281731 (+ 752us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":105,"threads_started":1}
I20260812 03:05:42.285454 27360 tablet_service.cc:1467] Processing CreateTablet for tablet c1829fce17a744c09e0dbc0fbfb8ee60 (DEFAULT_TABLE table=test-table [id=3a9060b2cf5246ce93597adf842d715d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:05:42.286697 27360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1829fce17a744c09e0dbc0fbfb8ee60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:05:42.295723 28024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:42.273202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:39639 (local address 127.0.0.1:38952)
0812 03:05:42.274640 (+ 1438us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:42.274653 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:42.274737 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:42.280813 (+ 6076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:42.280815 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:42.280828 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:05:42.280917 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:42.280922 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:42.289057 (+ 8135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:42.289060 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:42.290238 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:42.290250 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:42.295272 (+ 5022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:42.295277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:42.295370 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:42.295395 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 03:05:42.295473 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 03:05:42.295545 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1367,"thread_start_us":192,"threads_started":1}
I20260812 03:05:42.297394 28030 tablet_bootstrap.cc:492] T c1829fce17a744c09e0dbc0fbfb8ee60 P e28d76be67d8499ca9ce55410e6e432f: Bootstrap starting.
I20260812 03:05:42.297690 28022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:42.274104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:44157 (local address 127.0.0.1:41114)
0812 03:05:42.280611 (+ 6507us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:42.280625 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:42.280711 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:42.291197 (+ 10486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:42.291200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:42.291219 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:05:42.291308 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:42.291315 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:42.292727 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:42.292730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:42.293926 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:42.293938 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:42.297294 (+ 3356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:42.297298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:42.297391 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:42.297416 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 03:05:42.297494 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 03:05:42.297560 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6439,"thread_start_us":147,"threads_started":1}
I20260812 03:05:42.298323 28021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:42.273360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38952 (local address 127.5.96.130:39639)
0812 03:05:42.279865 (+ 6505us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:42.279872 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:42.279896 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:05:42.279977 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:42.279983 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:42.280211 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:42.280363 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:05:42.287742 (+ 7379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:42.288913 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:42.294081 (+ 5168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:42.294922 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:42.295034 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:42.295200 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:42.298060 (+ 2860us) server_negotiation.cc:299] Negotiation successful
0812 03:05:42.298179 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6394,"thread_start_us":91,"threads_started":1}
I20260812 03:05:42.299449 28025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:42.274556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41114 (local address 127.5.96.131:44157)
0812 03:05:42.290545 (+ 15989us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:42.290552 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:42.290578 (+ 26us) server_negotiation.cc:407] Connection header received
0812 03:05:42.290658 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:42.290667 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:42.290867 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:42.291019 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:05:42.291445 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:42.292588 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:42.296222 (+ 3634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:42.296959 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:42.297065 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:42.297223 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:42.299223 (+ 2000us) server_negotiation.cc:299] Negotiation successful
0812 03:05:42.299318 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15870,"thread_start_us":136,"threads_started":1}
I20260812 03:05:42.300630 28030 tablet_bootstrap.cc:654] T c1829fce17a744c09e0dbc0fbfb8ee60 P e28d76be67d8499ca9ce55410e6e432f: Neither blocks nor log segments found. Creating new log.
I20260812 03:05:42.301237 27592 tablet_service.cc:1467] Processing CreateTablet for tablet c1829fce17a744c09e0dbc0fbfb8ee60 (DEFAULT_TABLE table=test-table [id=3a9060b2cf5246ce93597adf842d715d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:05:42.302314 27592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1829fce17a744c09e0dbc0fbfb8ee60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:05:42.302413 27836 tablet_service.cc:1467] Processing CreateTablet for tablet c1829fce17a744c09e0dbc0fbfb8ee60 (DEFAULT_TABLE table=test-table [id=3a9060b2cf5246ce93597adf842d715d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:05:42.302799 28030 log.cc:824] T c1829fce17a744c09e0dbc0fbfb8ee60 P e28d76be67d8499ca9ce55410e6e432f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:05:42.303368 27836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1829fce17a744c09e0dbc0fbfb8ee60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:05:42.315213 28038 tablet_bootstrap.cc:492] T c1829fce17a744c09e0dbc0fbfb8ee60 P de300ba4028849a48302567a03a85cf3: Bootstrap starting.
I20260812 03:05:42.318002 28030 tablet_bootstrap.cc:492] T c1829fce17a744c09e0dbc0fbfb8ee60 P e28d76be67d8499ca9ce55410e6e432f: No bootstrap required, opened a new log
I20260812 03:05:42.318231 28030 ts_tablet_manager.cc:1397] T c1829fce17a744c09e0dbc0fbfb8ee60 P e28d76be67d8499ca9ce55410e6e432f: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.000s
I20260812 03:05:42.318266 28038 tablet_bootstrap.cc:654] T c1829fce17a744c09e0dbc0fbfb8ee60 P de300ba4028849a48302567a03a85cf3: Neither blocks nor log segments found. Creating new log.
I20260812 03:05:42.319289 28038 log.cc:824] T c1829fce17a744c09e0dbc0fbfb8ee60 P de300ba4028849a48302567a03a85cf3: Log is configured to *not* fsync() on all Append() calls
I20260812 03:05:42.320052 28035 tablet_bootstrap.cc:492] T c1829fce17a744c09e0dbc0fbfb8ee60 P 776402ce26e947669a65ccd4d40e97c4: Bootstrap starting.
I20260812 03:05:42.323174 28035 tablet_bootstrap.cc:654] T c1829fce17a744c09e0dbc0fbfb8ee60 P 776402ce26e947669a65ccd4d40e97c4: Neither blocks nor log segments found. Creating new log.
I20260812 03:05:42.324237 28035 log.cc:824] T c1829fce17a744c09e0dbc0fbfb8ee60 P 776402ce26e947669a65ccd4d40e97c4: Log is configured to *not* fsync() on all Append() calls
I20260812 03:05:42.324661 28030 raft_consensus.cc:348] T c1829fce17a744c09e0dbc0fbfb8ee60 P e28d76be67d8499ca9ce55410e6e432f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e28d76be67d8499ca9ce55410e6e432f" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39981 } } peers { permanent_uuid: "de300ba4028849a48302567a03a85cf3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 39639 } } peers { permanent_uuid: "776402ce26e947669a65ccd4d40e97c4" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44157 } }
I20260812 03:05:42.325528 28030 raft_consensus.cc:374] T c1829fce17a744c09e0dbc0fbfb8ee60 P e28d76be67d8499ca9ce55410e6e432f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:05:42.325623 28030 raft_consensus.cc:729] T c1829fce17a744c09e0dbc0fbfb8ee60 P e28d76be67d8499ca9ce55410e6e432f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e28d76be67d8499ca9ce55410e6e432f, State: Initialized, Role: FOLLOWER
I20260812 03:05:42.326390 28030 consensus_queue.cc:260] T c1829fce17a744c09e0dbc0fbfb8ee60 P e28d76be67d8499ca9ce55410e6e432f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e28d76be67d8499ca9ce55410e6e432f" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39981 } } peers { permanent_uuid: "de300ba4028849a48302567a03a85cf3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 39639 } } peers { permanent_uuid: "776402ce26e947669a65ccd4d40e97c4" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44157 } }
I20260812 03:05:42.328063 28030 ts_tablet_manager.cc:1428] T c1829fce17a744c09e0dbc0fbfb8ee60 P e28d76be67d8499ca9ce55410e6e432f: Time spent starting tablet: real 0.010s user 0.000s sys 0.007s
I20260812 03:05:42.328296 28038 tablet_bootstrap.cc:492] T c1829fce17a744c09e0dbc0fbfb8ee60 P de300ba4028849a48302567a03a85cf3: No bootstrap required, opened a new log
I20260812 03:05:42.328444 27501 heartbeater.cc:503] Master 127.5.96.190:40419 was elected leader, sending a full tablet report...
I20260812 03:05:42.328557 28038 ts_tablet_manager.cc:1397] T c1829fce17a744c09e0dbc0fbfb8ee60 P de300ba4028849a48302567a03a85cf3: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.002s
I20260812 03:05:42.333009 28038 raft_consensus.cc:348] T c1829fce17a744c09e0dbc0fbfb8ee60 P de300ba4028849a48302567a03a85cf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e28d76be67d8499ca9ce55410e6e432f" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39981 } } peers { permanent_uuid: "de300ba4028849a48302567a03a85cf3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 39639 } } peers { permanent_uuid: "776402ce26e947669a65ccd4d40e97c4" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44157 } }
I20260812 03:05:42.333324 28038 raft_consensus.cc:374] T c1829fce17a744c09e0dbc0fbfb8ee60 P de300ba4028849a48302567a03a85cf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:05:42.333410 28038 raft_consensus.cc:729] T c1829fce17a744c09e0dbc0fbfb8ee60 P de300ba4028849a48302567a03a85cf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de300ba4028849a48302567a03a85cf3, State: Initialized, Role: FOLLOWER
I20260812 03:05:42.333950 28038 consensus_queue.cc:260] T c1829fce17a744c09e0dbc0fbfb8ee60 P de300ba4028849a48302567a03a85cf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e28d76be67d8499ca9ce55410e6e432f" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39981 } } peers { permanent_uuid: "de300ba4028849a48302567a03a85cf3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 39639 } } peers { permanent_uuid: "776402ce26e947669a65ccd4d40e97c4" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44157 } }
I20260812 03:05:42.334982 28038 ts_tablet_manager.cc:1428] T c1829fce17a744c09e0dbc0fbfb8ee60 P de300ba4028849a48302567a03a85cf3: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
I20260812 03:05:42.335350 27742 heartbeater.cc:503] Master 127.5.96.190:40419 was elected leader, sending a full tablet report...
I20260812 03:05:42.345927 28035 tablet_bootstrap.cc:492] T c1829fce17a744c09e0dbc0fbfb8ee60 P 776402ce26e947669a65ccd4d40e97c4: No bootstrap required, opened a new log
I20260812 03:05:42.346213 28035 ts_tablet_manager.cc:1397] T c1829fce17a744c09e0dbc0fbfb8ee60 P 776402ce26e947669a65ccd4d40e97c4: Time spent bootstrapping tablet: real 0.026s user 0.006s sys 0.000s
I20260812 03:05:42.372589 28035 raft_consensus.cc:348] T c1829fce17a744c09e0dbc0fbfb8ee60 P 776402ce26e947669a65ccd4d40e97c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e28d76be67d8499ca9ce55410e6e432f" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39981 } } peers { permanent_uuid: "de300ba4028849a48302567a03a85cf3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 39639 } } peers { permanent_uuid: "776402ce26e947669a65ccd4d40e97c4" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44157 } }
I20260812 03:05:42.373011 28035 raft_consensus.cc:374] T c1829fce17a744c09e0dbc0fbfb8ee60 P 776402ce26e947669a65ccd4d40e97c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:05:42.373100 28035 raft_consensus.cc:729] T c1829fce17a744c09e0dbc0fbfb8ee60 P 776402ce26e947669a65ccd4d40e97c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776402ce26e947669a65ccd4d40e97c4, State: Initialized, Role: FOLLOWER
I20260812 03:05:42.373698 28035 consensus_queue.cc:260] T c1829fce17a744c09e0dbc0fbfb8ee60 P 776402ce26e947669a65ccd4d40e97c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e28d76be67d8499ca9ce55410e6e432f" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39981 } } peers { permanent_uuid: "de300ba4028849a48302567a03a85cf3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 39639 } } peers { permanent_uuid: "776402ce26e947669a65ccd4d40e97c4" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44157 } }
I20260812 03:05:42.375077 28035 ts_tablet_manager.cc:1428] T c1829fce17a744c09e0dbc0fbfb8ee60 P 776402ce26e947669a65ccd4d40e97c4: Time spent starting tablet: real 0.027s user 0.007s sys 0.000s
I20260812 03:05:42.375543 27978 heartbeater.cc:503] Master 127.5.96.190:40419 was elected leader, sending a full tablet report...
W20260812 03:05:42.448589 27979 tablet.cc:2369] T c1829fce17a744c09e0dbc0fbfb8ee60 P 776402ce26e947669a65ccd4d40e97c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:05:42.460801 27505 tablet.cc:2369] T c1829fce17a744c09e0dbc0fbfb8ee60 P e28d76be67d8499ca9ce55410e6e432f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:05:42.461601 27744 tablet.cc:2369] T c1829fce17a744c09e0dbc0fbfb8ee60 P de300ba4028849a48302567a03a85cf3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:05:42.597163 28042 raft_consensus.cc:482] T c1829fce17a744c09e0dbc0fbfb8ee60 P e28d76be67d8499ca9ce55410e6e432f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:05:42.597424 28042 raft_consensus.cc:504] T c1829fce17a744c09e0dbc0fbfb8ee60 P e28d76be67d8499ca9ce55410e6e432f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e28d76be67d8499ca9ce55410e6e432f" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39981 } } peers { permanent_uuid: "de300ba4028849a48302567a03a85cf3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 39639 } } peers { permanent_uuid: "776402ce26e947669a65ccd4d40e97c4" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44157 } }
I20260812 03:05:42.598861 28042 leader_election.cc:288] T c1829fce17a744c09e0dbc0fbfb8ee60 P e28d76be67d8499ca9ce55410e6e432f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de300ba4028849a48302567a03a85cf3 (127.5.96.130:39639), 776402ce26e947669a65ccd4d40e97c4 (127.5.96.131:44157)
I20260812 03:05:42.604668 28025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:42.599688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:47365 (local address 127.5.96.131:44157)
0812 03:05:42.599849 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:42.599854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:42.599977 (+ 123us) server_negotiation.cc:407] Connection header received
0812 03:05:42.600189 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:42.600192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:42.600241 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:42.600286 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:05:42.600803 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:42.601581 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:42.603288 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:42.603923 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:42.604018 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:42.604073 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:42.604458 (+ 385us) server_negotiation.cc:299] Negotiation successful
0812 03:05:42.604537 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260812 03:05:42.604729 28021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:42.599591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:53625 (local address 127.5.96.130:39639)
0812 03:05:42.599751 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:42.599756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:42.599933 (+ 177us) server_negotiation.cc:407] Connection header received
0812 03:05:42.600169 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:42.600173 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:42.600234 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:42.600281 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:05:42.600773 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:42.601568 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:42.603313 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:42.603946 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:42.604036 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:42.604083 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:42.604527 (+ 444us) server_negotiation.cc:299] Negotiation successful
0812 03:05:42.604602 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:05:42.605315 28058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:42.599114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:39639 (local address 127.5.96.129:53625)
0812 03:05:42.599819 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:42.599834 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:42.600039 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:42.600422 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:42.600424 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:42.600464 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:05:42.600644 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:42.600651 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:42.601708 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:42.601711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:42.603162 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:42.603174 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:42.604177 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:42.604180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:42.604380 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:42.604404 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:05:42.604912 (+ 508us) client_negotiation.cc:240] Negotiation successful
0812 03:05:42.604987 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":71,"spinlock_wait_cycles":26624,"thread_start_us":148,"threads_started":1}
I20260812 03:05:42.605343 28057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:42.599267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:44157 (local address 127.5.96.129:47365)
0812 03:05:42.599894 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:42.599907 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:42.600055 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:42.600425 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:42.600427 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:42.600466 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:05:42.600659 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:42.600666 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:42.601727 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:42.601730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:42.603145 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:42.603157 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:42.604179 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:42.604183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:42.604321 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:42.604343 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:05:42.605117 (+ 774us) client_negotiation.cc:240] Negotiation successful
0812 03:05:42.605192 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"mutex_wait_us":154,"spinlock_wait_cycles":8192,"thread_start_us":114,"threads_started":1}
I20260812 03:05:42.606106 27660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1829fce17a744c09e0dbc0fbfb8ee60" candidate_uuid: "e28d76be67d8499ca9ce55410e6e432f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de300ba4028849a48302567a03a85cf3" is_pre_election: true
I20260812 03:05:42.606436 27660 raft_consensus.cc:2454] T c1829fce17a744c09e0dbc0fbfb8ee60 P de300ba4028849a48302567a03a85cf3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e28d76be67d8499ca9ce55410e6e432f in term 0.
I20260812 03:05:42.607064 27284 leader_election.cc:302] T c1829fce17a744c09e0dbc0fbfb8ee60 P e28d76be67d8499ca9ce55410e6e432f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de300ba4028849a48302567a03a85cf3, e28d76be67d8499ca9ce55410e6e432f; no voters:
I20260812 03:05:42.607412 27873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1829fce17a744c09e0dbc0fbfb8ee60" candidate_uuid: "e28d76be67d8499ca9ce55410e6e432f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776402ce26e947669a65ccd4d40e97c4" is_pre_election: true
I20260812 03:05:42.607743 27873 raft_consensus.cc:2454] T c1829fce17a744c09e0dbc0fbfb8ee60 P 776402ce26e947669a65ccd4d40e97c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e28d76be67d8499ca9ce55410e6e432f in term 0.
I20260812 03:05:42.608044 28042 raft_consensus.cc:2781] T c1829fce17a744c09e0dbc0fbfb8ee60 P e28d76be67d8499ca9ce55410e6e432f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:05:42.608148 28042 raft_consensus.cc:482] T c1829fce17a744c09e0dbc0fbfb8ee60 P e28d76be67d8499ca9ce55410e6e432f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:05:42.608191 28042 raft_consensus.cc:3037] T c1829fce17a744c09e0dbc0fbfb8ee60 P e28d76be67d8499ca9ce55410e6e432f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:05:42.609513 28042 raft_consensus.cc:504] T c1829fce17a744c09e0dbc0fbfb8ee60 P e28d76be67d8499ca9ce55410e6e432f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e28d76be67d8499ca9ce55410e6e432f" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39981 } } peers { permanent_uuid: "de300ba4028849a48302567a03a85cf3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 39639 } } peers { permanent_uuid: "776402ce26e947669a65ccd4d40e97c4" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44157 } }
I20260812 03:05:42.609859 28042 leader_election.cc:288] T c1829fce17a744c09e0dbc0fbfb8ee60 P e28d76be67d8499ca9ce55410e6e432f [CANDIDATE]: Term 1 election: Requested vote from peers de300ba4028849a48302567a03a85cf3 (127.5.96.130:39639), 776402ce26e947669a65ccd4d40e97c4 (127.5.96.131:44157)
I20260812 03:05:42.610278 27660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1829fce17a744c09e0dbc0fbfb8ee60" candidate_uuid: "e28d76be67d8499ca9ce55410e6e432f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de300ba4028849a48302567a03a85cf3"
I20260812 03:05:42.610384 27873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1829fce17a744c09e0dbc0fbfb8ee60" candidate_uuid: "e28d76be67d8499ca9ce55410e6e432f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776402ce26e947669a65ccd4d40e97c4"
I20260812 03:05:42.610394 27660 raft_consensus.cc:3037] T c1829fce17a744c09e0dbc0fbfb8ee60 P de300ba4028849a48302567a03a85cf3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:05:42.610497 27873 raft_consensus.cc:3037] T c1829fce17a744c09e0dbc0fbfb8ee60 P 776402ce26e947669a65ccd4d40e97c4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:05:42.612016 27660 raft_consensus.cc:2454] T c1829fce17a744c09e0dbc0fbfb8ee60 P de300ba4028849a48302567a03a85cf3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e28d76be67d8499ca9ce55410e6e432f in term 1.
I20260812 03:05:42.612080 27873 raft_consensus.cc:2454] T c1829fce17a744c09e0dbc0fbfb8ee60 P 776402ce26e947669a65ccd4d40e97c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e28d76be67d8499ca9ce55410e6e432f in term 1.
I20260812 03:05:42.612404 27284 leader_election.cc:302] T c1829fce17a744c09e0dbc0fbfb8ee60 P e28d76be67d8499ca9ce55410e6e432f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de300ba4028849a48302567a03a85cf3, e28d76be67d8499ca9ce55410e6e432f; no voters:
I20260812 03:05:42.612661 28042 raft_consensus.cc:2781] T c1829fce17a744c09e0dbc0fbfb8ee60 P e28d76be67d8499ca9ce55410e6e432f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:05:42.612874 28042 raft_consensus.cc:686] T c1829fce17a744c09e0dbc0fbfb8ee60 P e28d76be67d8499ca9ce55410e6e432f [term 1 LEADER]: Becoming Leader. State: Replica: e28d76be67d8499ca9ce55410e6e432f, State: Running, Role: LEADER
I20260812 03:05:42.613219 28042 consensus_queue.cc:237] T c1829fce17a744c09e0dbc0fbfb8ee60 P e28d76be67d8499ca9ce55410e6e432f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e28d76be67d8499ca9ce55410e6e432f" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39981 } } peers { permanent_uuid: "de300ba4028849a48302567a03a85cf3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 39639 } } peers { permanent_uuid: "776402ce26e947669a65ccd4d40e97c4" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44157 } }
I20260812 03:05:42.616921 27123 catalog_manager.cc:5495] T c1829fce17a744c09e0dbc0fbfb8ee60 P e28d76be67d8499ca9ce55410e6e432f reported cstate change: term changed from 0 to 1, leader changed from <none> to e28d76be67d8499ca9ce55410e6e432f (127.5.96.129). New cstate: current_term: 1 leader_uuid: "e28d76be67d8499ca9ce55410e6e432f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e28d76be67d8499ca9ce55410e6e432f" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 39981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de300ba4028849a48302567a03a85cf3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 39639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776402ce26e947669a65ccd4d40e97c4" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44157 } health_report { overall_health: UNKNOWN } } }
I20260812 03:05:42.642663 28023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:42.637993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36410 (local address 127.5.96.129:39981)
0812 03:05:42.638106 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:42.638110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:42.638166 (+ 56us) server_negotiation.cc:407] Connection header received
0812 03:05:42.638336 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:42.638339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:42.638404 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:42.638513 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:05:42.638740 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:42.639521 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:42.640846 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:42.641236 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:42.641331 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:42.641682 (+ 351us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:05:42.642242 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:05:42.642363 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:42.642440 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 03:05:42.642499 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:05:42.649682 27873 raft_consensus.cc:1261] T c1829fce17a744c09e0dbc0fbfb8ee60 P 776402ce26e947669a65ccd4d40e97c4 [term 1 FOLLOWER]: Refusing update from remote peer e28d76be67d8499ca9ce55410e6e432f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:05:42.650431 28042 consensus_queue.cc:1035] T c1829fce17a744c09e0dbc0fbfb8ee60 P e28d76be67d8499ca9ce55410e6e432f [LEADER]: Connected to new peer: Peer: permanent_uuid: "776402ce26e947669a65ccd4d40e97c4" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:05:42.650784 27660 raft_consensus.cc:1261] T c1829fce17a744c09e0dbc0fbfb8ee60 P de300ba4028849a48302567a03a85cf3 [term 1 FOLLOWER]: Refusing update from remote peer e28d76be67d8499ca9ce55410e6e432f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:05:42.651451 28059 consensus_queue.cc:1035] T c1829fce17a744c09e0dbc0fbfb8ee60 P e28d76be67d8499ca9ce55410e6e432f [LEADER]: Connected to new peer: Peer: permanent_uuid: "de300ba4028849a48302567a03a85cf3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 39639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:05:42.659511 28064 mvcc.cc:205] Tried to move back new op lower bound from 7317520149085294592 to 7317520148946411520. Current Snapshot: MvccSnapshot[applied={T|T < 7317520149085294592}]
I20260812 03:05:42.663748 28068 mvcc.cc:205] Tried to move back new op lower bound from 7317520149085294592 to 7317520148946411520. Current Snapshot: MvccSnapshot[applied={T|T < 7317520149085294592}]
I20260812 03:05:42.666536 28065 mvcc.cc:205] Tried to move back new op lower bound from 7317520149085294592 to 7317520148946411520. Current Snapshot: MvccSnapshot[applied={T|T < 7317520149085294592}]
I20260812 03:05:42.673892 27121 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:05:42.674055 27121 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:05:42.676826 27121 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7f5f3ff51a5e4ccebeb4a5f3d8101ebf: Sending DeleteTablet for 3 replicas of tablet c1829fce17a744c09e0dbc0fbfb8ee60
I20260812 03:05:42.678161 27592 tablet_service.cc:1514] Processing DeleteTablet for tablet c1829fce17a744c09e0dbc0fbfb8ee60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:05:42 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:38952
I20260812 03:05:42.678779 27836 tablet_service.cc:1514] Processing DeleteTablet for tablet c1829fce17a744c09e0dbc0fbfb8ee60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:05:42 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:41114
I20260812 03:05:42.679726 27360 tablet_service.cc:1514] Processing DeleteTablet for tablet c1829fce17a744c09e0dbc0fbfb8ee60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:05:42 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:36400
I20260812 03:05:42.680833 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 27209
I20260812 03:05:42.693760 28078 tablet_replica.cc:330] T c1829fce17a744c09e0dbc0fbfb8ee60 P de300ba4028849a48302567a03a85cf3: stopping tablet replica
I20260812 03:05:42.694028 28079 tablet_replica.cc:330] T c1829fce17a744c09e0dbc0fbfb8ee60 P 776402ce26e947669a65ccd4d40e97c4: stopping tablet replica
I20260812 03:05:42.694470 28079 raft_consensus.cc:2229] T c1829fce17a744c09e0dbc0fbfb8ee60 P 776402ce26e947669a65ccd4d40e97c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:05:42.694844 28078 raft_consensus.cc:2229] T c1829fce17a744c09e0dbc0fbfb8ee60 P de300ba4028849a48302567a03a85cf3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:05:42.695261 28078 raft_consensus.cc:2258] T c1829fce17a744c09e0dbc0fbfb8ee60 P de300ba4028849a48302567a03a85cf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:05:42.695699 28079 raft_consensus.cc:2258] T c1829fce17a744c09e0dbc0fbfb8ee60 P 776402ce26e947669a65ccd4d40e97c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:05:42.697914 28078 ts_tablet_manager.cc:1905] T c1829fce17a744c09e0dbc0fbfb8ee60 P de300ba4028849a48302567a03a85cf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:05:42.700471 28079 ts_tablet_manager.cc:1905] T c1829fce17a744c09e0dbc0fbfb8ee60 P 776402ce26e947669a65ccd4d40e97c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:05:42.701370 28078 ts_tablet_manager.cc:1918] T c1829fce17a744c09e0dbc0fbfb8ee60 P de300ba4028849a48302567a03a85cf3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:05:42.701534 28078 log.cc:1196] T c1829fce17a744c09e0dbc0fbfb8ee60 P de300ba4028849a48302567a03a85cf3: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-1/wal/wals/c1829fce17a744c09e0dbc0fbfb8ee60
I20260812 03:05:42.701951 28078 ts_tablet_manager.cc:1939] T c1829fce17a744c09e0dbc0fbfb8ee60 P de300ba4028849a48302567a03a85cf3: Deleting consensus metadata
W20260812 03:05:42.702553 27091 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.96.129:39981 (error 108)
I20260812 03:05:42.703227 28079 ts_tablet_manager.cc:1918] T c1829fce17a744c09e0dbc0fbfb8ee60 P 776402ce26e947669a65ccd4d40e97c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:05:42.703397 28079 log.cc:1196] T c1829fce17a744c09e0dbc0fbfb8ee60 P 776402ce26e947669a65ccd4d40e97c4: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786503790637001-5506-0/minicluster-data/ts-2/wal/wals/c1829fce17a744c09e0dbc0fbfb8ee60
I20260812 03:05:42.703398 27092 catalog_manager.cc:4841] TS de300ba4028849a48302567a03a85cf3 (127.5.96.130:39639): tablet c1829fce17a744c09e0dbc0fbfb8ee60 (table test-table [id=3a9060b2cf5246ce93597adf842d715d]) successfully deleted
I20260812 03:05:42.703877 28079 ts_tablet_manager.cc:1939] T c1829fce17a744c09e0dbc0fbfb8ee60 P 776402ce26e947669a65ccd4d40e97c4: Deleting consensus metadata
I20260812 03:05:42.704506 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 27509
I20260812 03:05:42.704573 28022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:42.704269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:39981 (local address 127.0.0.1:36416)
0812 03:05:42.704405 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:42.704485 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.129:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 03:05:42.708809 27094 catalog_manager.cc:4841] TS 776402ce26e947669a65ccd4d40e97c4 (127.5.96.131:44157): tablet c1829fce17a744c09e0dbc0fbfb8ee60 (table test-table [id=3a9060b2cf5246ce93597adf842d715d]) successfully deleted
W20260812 03:05:42.710762 27091 catalog_manager.cc:4584] TS e28d76be67d8499ca9ce55410e6e432f (127.5.96.129:39981): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c1829fce17a744c09e0dbc0fbfb8ee60: Network error: Client connection negotiation failed: client connection to 127.5.96.129:39981: connect: Connection refused (error 111)
I20260812 03:05:42.716720 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 27747
I20260812 03:05:42.725298 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 27068
I20260812 03:05:42.756639 5506 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 03:05:43.103029 5506 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 03:05:43.103137 5506 mini_ranger.cc:66] Stopping Ranger...
I20260812 03:05:43.446923 5506 mini_ranger.cc:68] Stopped Ranger
2026-08-12 03:05:43.447 UTC [25384] LOG: received smart shutdown request
2026-08-12 03:05:43.450 UTC [25384] LOG: background worker "logical replication launcher" (PID 25399) exited with exit code 1
2026-08-12 03:05:43.451 UTC [25391] LOG: shutting down
2026-08-12 03:05:43.565 UTC [25384] LOG: database system is shut down
2026-08-12T03:05:43Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (46624 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:05:43 dist-test-slave-6bbx krb5kdc[28096](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:05:43 dist-test-slave-6bbx krb5kdc[28096](info): set up 2 sockets
Aug 12 03:05:43 dist-test-slave-6bbx krb5kdc[28096](info): commencing operation
krb5kdc: starting...
W20260812 03:05:44.812747 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:05:44 dist-test-slave-6bbx krb5kdc[28096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:05:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:05:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:05:44.924547 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:40853
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:40853,127.5.96.189:38235,127.5.96.188:40785
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:05:45.031289 28119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:05:45.031505 28119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:05:45.031543 28119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:05:45.035370 28119 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:05:45.035416 28119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:05:45.035436 28119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:05:45.035466 28119 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:05:45.035491 28119 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:05:45.040455 28119 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:40853,127.5.96.189:38235,127.5.96.188:40785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:40853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:05:45.041314 28119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:05:45.042229 28119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:05:45.050019 28125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:45.050079 28124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:45.050098 28119 server_base.cc:1034] running on GCE node
W20260812 03:05:45.050266 28128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:45.050846 28119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:05:45.051828 28119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:05:45.052975 28119 hybrid_clock.cc:648] HybridClock initialized: now 1786503945052956 us; error 33 us; skew 500 ppm
Aug 12 03:05:45 dist-test-slave-6bbx krb5kdc[28096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503945, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:05:45.058079 28119 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:05:45.058898 28119 webserver.cc:460] Webserver started at http://127.5.96.190:41019/ using document root <none> and password file <none>
I20260812 03:05:45.059412 28119 fs_manager.cc:359] Metadata directory not provided
I20260812 03:05:45.059474 28119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:05:45.059649 28119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:05:45.061550 28119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "6ec4c05a0c104cdf84d85e3cd7667252"
format_stamp: "Formatted at 2026-08-12 03:05:45 on dist-test-slave-6bbx"
server_key: "2679b9d84996771dbb2c73fde084aae5"
server_key_iv: "57a5fd897852bfab1eefaccf814de385"
server_key_version: "encryptionkey@0"
I20260812 03:05:45.062058 28119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "6ec4c05a0c104cdf84d85e3cd7667252"
format_stamp: "Formatted at 2026-08-12 03:05:45 on dist-test-slave-6bbx"
server_key: "2679b9d84996771dbb2c73fde084aae5"
server_key_iv: "57a5fd897852bfab1eefaccf814de385"
server_key_version: "encryptionkey@0"
I20260812 03:05:45.065480 28119 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 03:05:45.067584 28140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:45.068712 28119 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:05:45.068821 28119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "6ec4c05a0c104cdf84d85e3cd7667252"
format_stamp: "Formatted at 2026-08-12 03:05:45 on dist-test-slave-6bbx"
server_key: "2679b9d84996771dbb2c73fde084aae5"
server_key_iv: "57a5fd897852bfab1eefaccf814de385"
server_key_version: "encryptionkey@0"
I20260812 03:05:45.068907 28119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:05:45.085228 28119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:05:45.087574 28119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:05:45.087751 28119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:05:45.094156 28245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:40853 every 8 connection(s)
I20260812 03:05:45.094153 28119 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:40853
I20260812 03:05:45.095026 28119 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:05:45.097587 28246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:05:45.099699 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 28119
I20260812 03:05:45.099824 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:05:45.100078 5506 external_mini_cluster.cc:1428] Setting key 0c5393f263bc5d37910659d7caae80cf
I20260812 03:05:45.101171 28246 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40853 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.96.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.189@KRBTEST.COM" created.
Aug 12 03:05:45 dist-test-slave-6bbx krb5kdc[28096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503945, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:05:45.116798 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.103232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:40853 (local address 127.0.0.1:38368)
0812 03:05:45.103706 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:45.103754 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:45.104534 (+ 780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:45.105301 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:45.105306 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:45.105620 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:05:45.105852 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:45.105865 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.107434 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.107437 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:45.108191 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:45.108201 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.108984 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.108987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:45.109073 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:45.109568 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:05:45.109582 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:05:45.112542 (+ 2960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:05:45.114172 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:45.114180 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:45.114189 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:45.114401 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:45.114711 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:45.114714 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:45.114716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:45.114824 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:45.115337 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:05:45.115342 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:05:45.115466 (+ 124us) client_negotiation.cc:769] Sending connection context
0812 03:05:45.115594 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 03:05:45.115946 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":92,"spinlock_wait_cycles":13568,"thread_start_us":120,"threads_started":1}
I20260812 03:05:45.116806 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.103231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38368 (local address 127.5.96.190:40853)
0812 03:05:45.103598 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:45.103615 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:45.103854 (+ 239us) server_negotiation.cc:407] Connection header received
0812 03:05:45.104851 (+ 997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:45.104859 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:45.105102 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:45.105302 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:45.105998 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.107327 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.108310 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.108857 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.109027 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:45.112669 (+ 3642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:45.112674 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:45.112683 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:45.112702 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:45.114076 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.114505 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.114509 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.114510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.114628 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.114912 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.114916 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.114917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.115232 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:45.115343 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:45.115709 (+ 366us) server_negotiation.cc:299] Negotiation successful
0812 03:05:45.115884 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":244,"thread_start_us":102,"threads_started":1}
I20260812 03:05:45.119359 28246 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.189" port: 38235 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:05:45.119989 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.119762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:38235 (local address 127.0.0.1:59292)
0812 03:05:45.119851 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:45.119920 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.189:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 03:05:45.120162 28151 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.96.189:38235: connect: Connection refused (error 111)
I20260812 03:05:45.120910 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.120703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:38235 (local address 127.0.0.1:59296)
0812 03:05:45.120803 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:45.120852 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.189:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 03:05:45.121042 28246 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.96.189:38235: Network error: Client connection negotiation failed: client connection to 127.5.96.189:38235: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.189.keytab.
Entry for principal kudu/127.5.96.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.189.keytab.
I20260812 03:05:45.125998 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.189:38235
--webserver_interface=127.5.96.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.189.keytab
--principal=kudu/127.5.96.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:40853,127.5.96.189:38235,127.5.96.188:40785
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:05:45.170274 28246 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.96.189" port: 38235 } attempt: 1
I20260812 03:05:45.171079 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.170809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:38235 (local address 127.0.0.1:59310)
0812 03:05:45.170946 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:45.171005 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.189:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 03:05:45.171815 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.171563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:38235 (local address 127.0.0.1:59314)
0812 03:05:45.171704 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:45.171753 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.189:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 03:05:45.171939 28246 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.96.189:38235: Network error: Client connection negotiation failed: client connection to 127.5.96.189:38235: connect: Connection refused (error 111)
W20260812 03:05:45.232903 28254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:05:45.233115 28254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:05:45.233155 28254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:05:45.236979 28254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:05:45.237020 28254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:05:45.237039 28254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:05:45.237067 28254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:05:45.237093 28254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:05:45.240077 28246 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.96.189" port: 38235 } attempt: 2
I20260812 03:05:45.240787 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.240523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:38235 (local address 127.0.0.1:59324)
0812 03:05:45.240658 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:45.240715 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.189:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 03:05:45.241488 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.241296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:38235 (local address 127.0.0.1:59334)
0812 03:05:45.241385 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:45.241431 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.189:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 03:05:45.241607 28246 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.96.189:38235: Network error: Client connection negotiation failed: client connection to 127.5.96.189:38235: connect: Connection refused (error 111)
I20260812 03:05:45.242265 28254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:40853,127.5.96.189:38235,127.5.96.188:40785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.189.keytab
--principal=kudu/127.5.96.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.189:38235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.96.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:05:45.243136 28254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:05:45.244052 28254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:05:45.251528 28273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:45.251544 28275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:45.251600 28270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:45.251725 28254 server_base.cc:1034] running on GCE node
I20260812 03:05:45.252323 28254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:05:45.253250 28254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:05:45.254400 28254 hybrid_clock.cc:648] HybridClock initialized: now 1786503945254370 us; error 39 us; skew 500 ppm
Aug 12 03:05:45 dist-test-slave-6bbx krb5kdc[28096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503945, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:05:45.256945 28254 init.cc:377] Logged in from keytab as kudu/127.5.96.189@KRBTEST.COM (short username kudu)
I20260812 03:05:45.257774 28254 webserver.cc:460] Webserver started at http://127.5.96.189:46453/ using document root <none> and password file <none>
I20260812 03:05:45.258266 28254 fs_manager.cc:359] Metadata directory not provided
I20260812 03:05:45.258332 28254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:05:45.258519 28254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:05:45.260435 28254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-1/data/instance:
uuid: "9063ac395df54113ae19a1e57360d327"
format_stamp: "Formatted at 2026-08-12 03:05:45 on dist-test-slave-6bbx"
server_key: "12aae862987b0f78f2ea7823397348bc"
server_key_iv: "a8c46c84f19557e200c2e8d1cd066de4"
server_key_version: "encryptionkey@0"
I20260812 03:05:45.260969 28254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-1/wal/instance:
uuid: "9063ac395df54113ae19a1e57360d327"
format_stamp: "Formatted at 2026-08-12 03:05:45 on dist-test-slave-6bbx"
server_key: "12aae862987b0f78f2ea7823397348bc"
server_key_iv: "a8c46c84f19557e200c2e8d1cd066de4"
server_key_version: "encryptionkey@0"
I20260812 03:05:45.264322 28254 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:05:45.266409 28288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:45.267474 28254 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:05:45.267582 28254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-1/wal
uuid: "9063ac395df54113ae19a1e57360d327"
format_stamp: "Formatted at 2026-08-12 03:05:45 on dist-test-slave-6bbx"
server_key: "12aae862987b0f78f2ea7823397348bc"
server_key_iv: "a8c46c84f19557e200c2e8d1cd066de4"
server_key_version: "encryptionkey@0"
I20260812 03:05:45.267686 28254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:05:45.284780 28254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:05:45.287106 28254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:05:45.287267 28254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:05:45.293805 28254 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.189:38235
I20260812 03:05:45.293818 28378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.189:38235 every 8 connection(s)
I20260812 03:05:45.294636 28254 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-1/data/info.pb
I20260812 03:05:45.297137 28380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:05:45.300575 28380 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40853 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:05:45.301285 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 28254
I20260812 03:05:45.301404 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-1/wal/instance
I20260812 03:05:45.301632 5506 external_mini_cluster.cc:1428] Setting key 3880c248b2512552d8c0520913596296
Aug 12 03:05:45 dist-test-slave-6bbx krb5kdc[28096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503945, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.189@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.96.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.188@KRBTEST.COM" created.
I20260812 03:05:45.313256 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.302286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38380 (local address 127.5.96.190:40853)
0812 03:05:45.302397 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:45.302400 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:45.303298 (+ 898us) server_negotiation.cc:407] Connection header received
0812 03:05:45.304360 (+ 1062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:45.304363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:45.304417 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:45.304479 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:45.305711 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.306222 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.307334 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.307566 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.307654 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:45.309820 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:45.309823 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:45.309825 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:45.309848 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:45.311097 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.311545 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.311548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.311550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.311639 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.311979 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.311982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.311984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.312181 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:45.312372 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:45.312964 (+ 592us) server_negotiation.cc:299] Negotiation successful
0812 03:05:45.313097 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:05:45.313863 28382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.302717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:40853 (local address 127.0.0.1:38380)
0812 03:05:45.303165 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:45.303203 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:45.304120 (+ 917us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:45.304666 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:45.304675 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:45.305107 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:05:45.305573 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:45.305585 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.306347 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.306350 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:45.307214 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:45.307224 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.307724 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.307729 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:45.307885 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:45.308365 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:05:45.308381 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:05:45.309670 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:05:45.311201 (+ 1531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:45.311207 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:45.311220 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:45.311452 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:45.311767 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:45.311770 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:45.311772 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:45.311904 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:45.312411 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:05:45.312418 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:05:45.312700 (+ 282us) client_negotiation.cc:769] Sending connection context
0812 03:05:45.312859 (+ 159us) client_negotiation.cc:240] Negotiation successful
0812 03:05:45.313086 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":131,"threads_started":1}
I20260812 03:05:45.315143 28380 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.189" port: 38235 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:05:45 dist-test-slave-6bbx krb5kdc[28096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503945, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.189@KRBTEST.COM for kudu/127.5.96.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.188.keytab.
Entry for principal kudu/127.5.96.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.188.keytab.
I20260812 03:05:45.325601 28382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.315576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:38235 (local address 127.0.0.1:59342)
0812 03:05:45.315766 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:45.315780 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:45.315857 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:45.316422 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:45.316424 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:45.316545 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:05:45.316628 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:45.316633 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.317962 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.317964 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:45.318479 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:45.318487 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.319268 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.319273 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:45.319374 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:45.319854 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:05:45.319870 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:05:45.322584 (+ 2714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:05:45.324272 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:45.324276 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:45.324277 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:45.324469 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:45.324695 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:45.324697 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:45.324699 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:45.324823 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:45.325200 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:05:45.325203 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:05:45.325287 (+ 84us) client_negotiation.cc:769] Sending connection context
0812 03:05:45.325377 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 03:05:45.325478 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128}
I20260812 03:05:45.325917 28385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.315700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59342 (local address 127.5.96.189:38235)
0812 03:05:45.316038 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:45.316051 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:45.316074 (+ 23us) server_negotiation.cc:407] Connection header received
0812 03:05:45.316116 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:45.316121 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:45.316300 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:45.316435 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:45.316732 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.317847 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.318595 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.319141 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.319251 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:45.322716 (+ 3465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:45.322733 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:45.322743 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:45.322764 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:45.324162 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.324551 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.324556 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.324557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.324626 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.324891 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.324893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.324894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.325121 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:45.325219 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:45.325463 (+ 244us) server_negotiation.cc:299] Negotiation successful
0812 03:05:45.325600 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":131,"threads_started":1}
I20260812 03:05:45.326328 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.188:40785
--webserver_interface=127.5.96.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.188.keytab
--principal=kudu/127.5.96.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:40853,127.5.96.189:38235,127.5.96.188:40785
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:05:45.327533 28380 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:05:45.328200 28382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.327998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:40785 (local address 127.0.0.1:34832)
0812 03:05:45.328089 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:45.328151 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 03:05:45.328382 28290 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.96.188:40785: connect: Connection refused (error 111)
I20260812 03:05:45.329144 28382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.328961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:40785 (local address 127.0.0.1:34838)
0812 03:05:45.329057 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:45.329098 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 03:05:45.329273 28380 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.96.188:40785: Network error: Client connection negotiation failed: client connection to 127.5.96.188:40785: connect: Connection refused (error 111)
I20260812 03:05:45.332754 28246 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.96.189" port: 38235 } attempt: 3
Aug 12 03:05:45 dist-test-slave-6bbx krb5kdc[28096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503945, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for kudu/127.5.96.189@KRBTEST.COM
I20260812 03:05:45.341296 28385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.333210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59350 (local address 127.5.96.189:38235)
0812 03:05:45.333312 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:45.333315 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:45.333326 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:05:45.333424 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:45.333426 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:45.333457 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:45.333514 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:45.334380 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.335423 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.336486 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.336702 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.336771 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:45.338671 (+ 1900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:45.338674 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:45.338676 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:45.338697 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:45.339943 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.340379 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.340382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.340383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.340449 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.340675 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.340677 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.340678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.340819 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:45.340883 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:45.341114 (+ 231us) server_negotiation.cc:299] Negotiation successful
0812 03:05:45.341192 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:05:45.341354 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.333128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:38235 (local address 127.0.0.1:59350)
0812 03:05:45.333243 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:45.333266 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:45.333348 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:45.333550 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:45.333553 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:45.333719 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:05:45.334280 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:45.334291 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.335534 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.335537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:45.336382 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:45.336394 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.336790 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.336794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:45.336898 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:45.337313 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:05:45.337327 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:05:45.338553 (+ 1226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:05:45.340050 (+ 1497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:45.340053 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:45.340055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:45.340298 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:45.340521 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:45.340524 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:45.340526 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:45.340606 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:45.340887 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:05:45.340890 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:05:45.340982 (+ 92us) client_negotiation.cc:769] Sending connection context
0812 03:05:45.341070 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 03:05:45.341189 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 03:05:45.342228 28246 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:05:45.342854 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.342635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:40785 (local address 127.0.0.1:34848)
0812 03:05:45.342739 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:45.342791 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 03:05:45.343501 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.343281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:40785 (local address 127.0.0.1:34854)
0812 03:05:45.343399 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:45.343445 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260812 03:05:45.343609 28246 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.96.188:40785: Network error: Client connection negotiation failed: client connection to 127.5.96.188:40785: connect: Connection refused (error 111)
I20260812 03:05:45.374730 28246 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } attempt: 1
I20260812 03:05:45.375361 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.375098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:40785 (local address 127.0.0.1:34862)
0812 03:05:45.375222 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:45.375287 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 03:05:45.375957 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.375762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:40785 (local address 127.0.0.1:34868)
0812 03:05:45.375852 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:45.375901 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 03:05:45.376075 28246 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.96.188:40785: Network error: Client connection negotiation failed: client connection to 127.5.96.188:40785: connect: Connection refused (error 111)
I20260812 03:05:45.378484 28380 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } attempt: 1
I20260812 03:05:45.379024 28382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.378806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:40785 (local address 127.0.0.1:34882)
0812 03:05:45.378919 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:45.378965 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 03:05:45.379652 28382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.379471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:40785 (local address 127.0.0.1:34886)
0812 03:05:45.379561 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:45.379602 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 03:05:45.379824 28380 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.96.188:40785: Network error: Client connection negotiation failed: client connection to 127.5.96.188:40785: connect: Connection refused (error 111)
W20260812 03:05:45.436496 28386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:05:45.436721 28386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:05:45.436774 28386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:05:45.440665 28386 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:05:45.440713 28386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:05:45.440734 28386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:05:45.440764 28386 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:05:45.440788 28386 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:05:45.445771 28386 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:40853,127.5.96.189:38235,127.5.96.188:40785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.188.keytab
--principal=kudu/127.5.96.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.188:40785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.96.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:05:45.446687 28386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:05:45.447614 28386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:05:45.447947 28380 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } attempt: 2
I20260812 03:05:45.448643 28382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.448386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:40785 (local address 127.0.0.1:34890)
0812 03:05:45.448517 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:45.448572 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 03:05:45.449344 28382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.449143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:40785 (local address 127.0.0.1:34898)
0812 03:05:45.449244 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:45.449290 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 03:05:45.449453 28380 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.96.188:40785: Network error: Client connection negotiation failed: client connection to 127.5.96.188:40785: connect: Connection refused (error 111)
I20260812 03:05:45.451228 28246 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } attempt: 2
I20260812 03:05:45.452095 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.451753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:40785 (local address 127.0.0.1:34906)
0812 03:05:45.451954 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:45.452017 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260812 03:05:45.452719 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.452528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:40785 (local address 127.0.0.1:34912)
0812 03:05:45.452619 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:45.452662 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 03:05:45.452832 28246 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.96.188:40785: Network error: Client connection negotiation failed: client connection to 127.5.96.188:40785: connect: Connection refused (error 111)
W20260812 03:05:45.455952 28400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:45.455947 28396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:45.455948 28393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:45.456138 28386 server_base.cc:1034] running on GCE node
I20260812 03:05:45.456821 28386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:05:45.457837 28386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:05:45.458992 28386 hybrid_clock.cc:648] HybridClock initialized: now 1786503945458977 us; error 23 us; skew 500 ppm
Aug 12 03:05:45 dist-test-slave-6bbx krb5kdc[28096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503945, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:05:45.461584 28386 init.cc:377] Logged in from keytab as kudu/127.5.96.188@KRBTEST.COM (short username kudu)
I20260812 03:05:45.462410 28386 webserver.cc:460] Webserver started at http://127.5.96.188:39453/ using document root <none> and password file <none>
I20260812 03:05:45.462893 28386 fs_manager.cc:359] Metadata directory not provided
I20260812 03:05:45.462951 28386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:05:45.463141 28386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:05:45.465029 28386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-2/data/instance:
uuid: "5531d5a1827945848cda591f581267f3"
format_stamp: "Formatted at 2026-08-12 03:05:45 on dist-test-slave-6bbx"
server_key: "5957822658f6495627cf7d62d782240c"
server_key_iv: "38c955431287108ec5c9e8a49a9fa5fc"
server_key_version: "encryptionkey@0"
I20260812 03:05:45.465544 28386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-2/wal/instance:
uuid: "5531d5a1827945848cda591f581267f3"
format_stamp: "Formatted at 2026-08-12 03:05:45 on dist-test-slave-6bbx"
server_key: "5957822658f6495627cf7d62d782240c"
server_key_iv: "38c955431287108ec5c9e8a49a9fa5fc"
server_key_version: "encryptionkey@0"
I20260812 03:05:45.468842 28386 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:05:45.470994 28413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:45.472218 28386 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:05:45.472328 28386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-2/wal
uuid: "5531d5a1827945848cda591f581267f3"
format_stamp: "Formatted at 2026-08-12 03:05:45 on dist-test-slave-6bbx"
server_key: "5957822658f6495627cf7d62d782240c"
server_key_iv: "38c955431287108ec5c9e8a49a9fa5fc"
server_key_version: "encryptionkey@0"
I20260812 03:05:45.472416 28386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:05:45.484661 28386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:05:45.486972 28386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:05:45.487103 28386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:05:45.493425 28386 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.188:40785
I20260812 03:05:45.493439 28506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.188:40785 every 8 connection(s)
I20260812 03:05:45.494257 28386 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-2/data/info.pb
I20260812 03:05:45.496796 28507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:05:45.500180 28507 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40853 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:05:45.501909 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 28386
I20260812 03:05:45.502023 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786503790637001-5506-0/minicluster-data/master-2/wal/instance
I20260812 03:05:45.502281 5506 external_mini_cluster.cc:1428] Setting key 737da80c72dc637c0de55748fda80e26
Aug 12 03:05:45 dist-test-slave-6bbx krb5kdc[28096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503945, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.188@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
Aug 12 03:05:45 dist-test-slave-6bbx krb5kdc[28096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.188@KRBTEST.COM
I20260812 03:05:45.512530 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.501830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38382 (local address 127.5.96.190:40853)
0812 03:05:45.501969 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:45.501973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:45.502781 (+ 808us) server_negotiation.cc:407] Connection header received
0812 03:05:45.503837 (+ 1056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:45.503840 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:45.503893 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:45.503960 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:45.504685 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.505186 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.506103 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.506321 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.506406 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:45.508827 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:45.508830 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:45.508832 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:45.508854 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:45.510341 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.510884 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.510888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.510889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.510976 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.511316 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.511319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.511321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.511521 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:45.511614 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:45.512111 (+ 497us) server_negotiation.cc:299] Negotiation successful
0812 03:05:45.512233 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:05:45.513125 28510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.502193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:40853 (local address 127.0.0.1:38382)
0812 03:05:45.502622 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:45.502671 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:45.503623 (+ 952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:45.504019 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:45.504024 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:45.504333 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:05:45.504556 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:45.504568 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.505283 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.505286 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:45.505994 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:45.506002 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.506449 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.506453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:45.506615 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:45.507100 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:05:45.507121 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:05:45.508664 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:05:45.510440 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:45.510448 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:45.510455 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:45.510792 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:45.511059 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:45.511063 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:45.511065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:45.511233 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:45.511620 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:05:45.511625 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:05:45.511893 (+ 268us) client_negotiation.cc:769] Sending connection context
0812 03:05:45.512053 (+ 160us) client_negotiation.cc:240] Negotiation successful
0812 03:05:45.512327 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"spinlock_wait_cycles":3456,"thread_start_us":119,"threads_started":1}
I20260812 03:05:45.513136 28515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.502856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34920 (local address 127.5.96.188:40785)
0812 03:05:45.503215 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:45.503223 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:45.503256 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:05:45.503623 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:45.503637 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:45.503858 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:45.504022 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:45.504796 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.505593 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.506398 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.506735 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.506825 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:45.509240 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:45.509246 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:45.509258 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:45.509284 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:45.510816 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.511228 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.511233 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.511234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.511314 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.511563 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.511566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.511568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.511872 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:45.512006 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:45.512258 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 03:05:45.512382 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":116,"threads_started":1}
I20260812 03:05:45.514195 28507 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.189" port: 38235 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:05:45 dist-test-slave-6bbx krb5kdc[28096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503945, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.188@KRBTEST.COM for kudu/127.5.96.189@KRBTEST.COM
Aug 12 03:05:45 dist-test-slave-6bbx krb5kdc[28096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:05:45.521560 28510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.514605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:38235 (local address 127.0.0.1:59354)
0812 03:05:45.514696 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:45.514709 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:45.514773 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:45.515053 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:45.515056 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:45.515177 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:05:45.515269 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:45.515276 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.516085 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.516087 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:45.516555 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:45.516561 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.517010 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.517014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:45.517108 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:45.517497 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:05:45.517508 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:05:45.518565 (+ 1057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:05:45.520215 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:45.520218 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:45.520220 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:45.520423 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:45.520692 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:45.520694 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:45.520696 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:45.520777 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:45.521113 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:05:45.521116 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:05:45.521208 (+ 92us) client_negotiation.cc:769] Sending connection context
0812 03:05:45.521318 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 03:05:45.521422 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 03:05:45.521577 28385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.514714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59354 (local address 127.5.96.189:38235)
0812 03:05:45.514844 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:45.514849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:45.514862 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:05:45.514905 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:45.514908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:45.514951 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:45.515013 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:45.515424 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.515970 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.516662 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.516901 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.516980 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:45.518688 (+ 1708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:45.518691 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:45.518693 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:45.518714 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:45.520114 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.520514 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.520517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.520518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.520591 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.520865 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.520867 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.520868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.521025 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:45.521095 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:45.521354 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 03:05:45.521438 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 03:05:45.522126 28507 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:05:45.523185 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.515333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38398 (local address 127.5.96.190:40853)
0812 03:05:45.515431 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:45.515435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:45.515447 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:05:45.515503 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:45.515506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:45.515542 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:45.515604 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:45.516162 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.516614 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.517433 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.517658 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.517733 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:45.520017 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:45.520020 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:45.520021 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:45.520042 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:45.521424 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.521848 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.521852 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.521854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.521936 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.522227 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.522230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.522232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.522410 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:45.522614 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:45.522859 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 03:05:45.523019 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
Aug 12 03:05:45 dist-test-slave-6bbx krb5kdc[28096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503945, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.188@KRBTEST.COM for kudu/127.5.96.188@KRBTEST.COM
Aug 12 03:05:45 dist-test-slave-6bbx krb5kdc[28096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.189@KRBTEST.COM
I20260812 03:05:45.529981 28515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.522618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34934 (local address 127.5.96.188:40785)
0812 03:05:45.522698 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:45.522702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:45.522966 (+ 264us) server_negotiation.cc:407] Connection header received
0812 03:05:45.522992 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:45.522994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:45.523027 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:45.523081 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:45.523488 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.524178 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.525063 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.525301 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.525381 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:45.527037 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:45.527040 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:45.527043 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:45.527065 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:45.528595 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.528940 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.528943 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.528945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.529009 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.529252 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.529255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.529256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.529408 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:45.529483 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:45.529745 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 03:05:45.529855 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":32}
I20260812 03:05:45.529990 28510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.522492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:40785 (local address 127.0.0.1:34934)
0812 03:05:45.522809 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:45.522823 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:45.522899 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:45.523162 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:45.523165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:45.523310 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:05:45.523388 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:45.523394 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.524265 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.524268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:45.524970 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:45.524981 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.525389 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.525393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:45.525461 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:45.525838 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:05:45.525852 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:05:45.526925 (+ 1073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:05:45.528689 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:45.528692 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:45.528693 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:45.528860 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:45.529084 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:45.529087 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:45.529089 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:45.529158 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:45.529495 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:05:45.529497 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:05:45.529585 (+ 88us) client_negotiation.cc:769] Sending connection context
0812 03:05:45.529682 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 03:05:45.529805 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260812 03:05:45.532816 28385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.524857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59362 (local address 127.5.96.189:38235)
0812 03:05:45.524958 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:45.524962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:45.525813 (+ 851us) server_negotiation.cc:407] Connection header received
0812 03:05:45.525855 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:45.525858 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:45.525899 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:45.525978 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:45.526484 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.526966 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.527833 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.528033 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.528107 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:45.530173 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:45.530198 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:45.530201 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:45.530228 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:45.531450 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.531877 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.531879 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.531881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.531945 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.532171 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.532173 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.532174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.532312 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:45.532461 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:45.532627 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 03:05:45.532710 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:05:45.533448 28507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5531d5a1827945848cda591f581267f3: Bootstrap starting.
I20260812 03:05:45.535771 5506 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:05:45.536028 28507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5531d5a1827945848cda591f581267f3: Neither blocks nor log segments found. Creating new log.
I20260812 03:05:45.536783 28507 log.cc:824] T 00000000000000000000000000000000 P 5531d5a1827945848cda591f581267f3: Log is configured to *not* fsync() on all Append() calls
I20260812 03:05:45.539361 28507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5531d5a1827945848cda591f581267f3: No bootstrap required, opened a new log
I20260812 03:05:45.540588 28380 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } attempt: 3
I20260812 03:05:45.543918 28507 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5531d5a1827945848cda591f581267f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ec4c05a0c104cdf84d85e3cd7667252" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40853 } } peers { permanent_uuid: "9063ac395df54113ae19a1e57360d327" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 38235 } } peers { permanent_uuid: "5531d5a1827945848cda591f581267f3" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } }
I20260812 03:05:45.544202 28507 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5531d5a1827945848cda591f581267f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:05:45.544271 28507 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5531d5a1827945848cda591f581267f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5531d5a1827945848cda591f581267f3, State: Initialized, Role: FOLLOWER
I20260812 03:05:45.544782 28507 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5531d5a1827945848cda591f581267f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ec4c05a0c104cdf84d85e3cd7667252" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40853 } } peers { permanent_uuid: "9063ac395df54113ae19a1e57360d327" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 38235 } } peers { permanent_uuid: "5531d5a1827945848cda591f581267f3" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } }
I20260812 03:05:45.546888 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.537673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38406 (local address 127.5.96.190:40853)
0812 03:05:45.537909 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:45.537913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:45.538040 (+ 127us) server_negotiation.cc:407] Connection header received
0812 03:05:45.538228 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:45.538231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:45.538273 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:45.538322 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:45.539115 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.539839 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.541158 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.541490 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.541566 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:45.543161 (+ 1595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:45.543165 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:45.543168 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:45.543206 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:45.544716 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.545196 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.545199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.545201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.545283 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.545926 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.545930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.545932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.546148 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:45.546251 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:45.546621 (+ 370us) server_negotiation.cc:299] Negotiation successful
0812 03:05:45.546750 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170}
I20260812 03:05:45.547168 28385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.538237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59366 (local address 127.5.96.189:38235)
0812 03:05:45.538350 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:45.538354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:45.538631 (+ 277us) server_negotiation.cc:407] Connection header received
0812 03:05:45.538813 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:45.538816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:45.538857 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:45.538902 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:45.539863 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.540561 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.541622 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.541928 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.541999 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:45.543479 (+ 1480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:45.543483 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:45.543485 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:45.543507 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:45.545201 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.545912 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.545915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.545917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.545999 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.546293 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.546296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.546298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.546484 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:45.546593 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:45.546931 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 03:05:45.547031 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 03:05:45.547580 28507 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5531d5a1827945848cda591f581267f3 [sys.catalog]: configured and running, proceeding with master startup.
Aug 12 03:05:45 dist-test-slave-6bbx krb5kdc[28096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503945, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.189@KRBTEST.COM for kudu/127.5.96.188@KRBTEST.COM
I20260812 03:05:45.550186 28528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5531d5a1827945848cda591f581267f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ec4c05a0c104cdf84d85e3cd7667252" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40853 } } peers { permanent_uuid: "9063ac395df54113ae19a1e57360d327" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 38235 } } peers { permanent_uuid: "5531d5a1827945848cda591f581267f3" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } } }
I20260812 03:05:45.551992 28515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.537790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34948 (local address 127.5.96.188:40785)
0812 03:05:45.537891 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:45.537894 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:45.538133 (+ 239us) server_negotiation.cc:407] Connection header received
0812 03:05:45.538293 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:45.538296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:45.538334 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:45.538378 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:45.539253 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.539928 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.541273 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.541601 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.541673 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:45.544730 (+ 3057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:45.544734 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:45.544736 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:45.544760 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:45.546314 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.550068 (+ 3754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.550071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.550073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.550153 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.550486 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.550489 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.550491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.550852 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:45.550977 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:45.551755 (+ 778us) server_negotiation.cc:299] Negotiation successful
0812 03:05:45.551857 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 03:05:45.551970 28246 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } attempt: 3
W20260812 03:05:45.553686 28547 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5531d5a1827945848cda591f581267f3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:05:45.553802 28547 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:05:45.554065 28382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.540933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:40785 (local address 127.0.0.1:34960)
0812 03:05:45.541021 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:45.541035 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:45.541116 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:45.544025 (+ 2909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:45.544028 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:45.544297 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:05:45.544388 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:45.544395 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.546355 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.546358 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:45.547053 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:45.547062 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.547650 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.547653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:45.547763 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:45.548224 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:05:45.548238 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:05:45.549552 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:05:45.551813 (+ 2261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:45.551817 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:45.551819 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:45.552046 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:45.553138 (+ 1092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:45.553140 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:45.553142 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:45.553252 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:45.553667 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:05:45.553671 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:05:45.553764 (+ 93us) client_negotiation.cc:769] Sending connection context
0812 03:05:45.553848 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 03:05:45.553964 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260812 03:05:45.551239 28386 master_runner.cc:186] Error getting master registration for 127.5.96.190:40853: OK, instance_id { permanent_uuid: "6ec4c05a0c104cdf84d85e3cd7667252" instance_seqno: 1786503945092402 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:05:45.554984 28386 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:05:45.550318 28528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5531d5a1827945848cda591f581267f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:05:45.555470 28527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.540984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34960 (local address 127.5.96.188:40785)
0812 03:05:45.542152 (+ 1168us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:45.542161 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:45.543448 (+ 1287us) server_negotiation.cc:407] Connection header received
0812 03:05:45.543479 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:45.543481 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:45.543551 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:45.544091 (+ 540us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:45.545533 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.546248 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.547197 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.547529 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.547954 (+ 425us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:45.549715 (+ 1761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:45.549718 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:45.549721 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:45.549743 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:45.551636 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.552172 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.552175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.552190 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.552276 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.553367 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.553370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.553371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.553557 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:45.555121 (+ 1564us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:45.555225 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 03:05:45.555333 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1049,"thread_start_us":92,"threads_started":1}
Aug 12 03:05:45 dist-test-slave-6bbx krb5kdc[28096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503945, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for kudu/127.5.96.188@KRBTEST.COM
I20260812 03:05:45.558012 28380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327: Bootstrap starting.
I20260812 03:05:45.560372 28380 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327: Neither blocks nor log segments found. Creating new log.
I20260812 03:05:45.560909 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.552391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:40785 (local address 127.0.0.1:34972)
0812 03:05:45.552627 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:45.552641 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:45.552736 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:45.553022 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:45.553025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:45.553201 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:05:45.553289 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:45.553295 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.554237 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.554240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:45.554977 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:45.554988 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.555586 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.555590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:45.555693 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:45.556115 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:05:45.556130 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:05:45.557483 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:05:45.559319 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:45.559323 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:45.559325 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:45.559557 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:45.559905 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:45.559908 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:45.559909 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:45.560000 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:45.560448 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:05:45.560451 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:05:45.560544 (+ 93us) client_negotiation.cc:769] Sending connection context
0812 03:05:45.560641 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 03:05:45.560761 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170}
I20260812 03:05:45.561167 28380 log.cc:824] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327: Log is configured to *not* fsync() on all Append() calls
I20260812 03:05:45.561345 28515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:45.552407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34972 (local address 127.5.96.188:40785)
0812 03:05:45.552496 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:45.552499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:45.552706 (+ 207us) server_negotiation.cc:407] Connection header received
0812 03:05:45.552820 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:45.552823 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:45.552861 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:45.552925 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:45.553421 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.554106 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.555122 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:45.555390 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:45.555481 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:45.557629 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:45.557633 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:45.557635 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:45.557658 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:45.559192 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.559705 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.559708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.559710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.559790 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:45.560114 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:45.560118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:45.560119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:45.560311 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:45.560400 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:45.561109 (+ 709us) server_negotiation.cc:299] Negotiation successful
0812 03:05:45.561213 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 03:05:45.563436 28380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327: No bootstrap required, opened a new log
I20260812 03:05:45.564038 28246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ec4c05a0c104cdf84d85e3cd7667252: Bootstrap starting.
I20260812 03:05:45.566409 28246 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ec4c05a0c104cdf84d85e3cd7667252: Neither blocks nor log segments found. Creating new log.
I20260812 03:05:45.567219 28246 log.cc:824] T 00000000000000000000000000000000 P 6ec4c05a0c104cdf84d85e3cd7667252: Log is configured to *not* fsync() on all Append() calls
I20260812 03:05:45.567526 28380 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ec4c05a0c104cdf84d85e3cd7667252" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40853 } } peers { permanent_uuid: "9063ac395df54113ae19a1e57360d327" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 38235 } } peers { permanent_uuid: "5531d5a1827945848cda591f581267f3" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } }
I20260812 03:05:45.567749 28380 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:05:45.567806 28380 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9063ac395df54113ae19a1e57360d327, State: Initialized, Role: FOLLOWER
I20260812 03:05:45.568281 28380 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ec4c05a0c104cdf84d85e3cd7667252" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40853 } } peers { permanent_uuid: "9063ac395df54113ae19a1e57360d327" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 38235 } } peers { permanent_uuid: "5531d5a1827945848cda591f581267f3" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } }
I20260812 03:05:45.569559 28380 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:05:45.569578 28555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ec4c05a0c104cdf84d85e3cd7667252" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40853 } } peers { permanent_uuid: "9063ac395df54113ae19a1e57360d327" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 38235 } } peers { permanent_uuid: "5531d5a1827945848cda591f581267f3" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } } }
I20260812 03:05:45.569762 28555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:05:45.569972 28246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ec4c05a0c104cdf84d85e3cd7667252: No bootstrap required, opened a new log
I20260812 03:05:45.573902 28246 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ec4c05a0c104cdf84d85e3cd7667252 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ec4c05a0c104cdf84d85e3cd7667252" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40853 } } peers { permanent_uuid: "9063ac395df54113ae19a1e57360d327" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 38235 } } peers { permanent_uuid: "5531d5a1827945848cda591f581267f3" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } }
I20260812 03:05:45.574105 28246 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ec4c05a0c104cdf84d85e3cd7667252 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:05:45.574158 28246 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ec4c05a0c104cdf84d85e3cd7667252 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ec4c05a0c104cdf84d85e3cd7667252, State: Initialized, Role: FOLLOWER
I20260812 03:05:45.574559 28246 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ec4c05a0c104cdf84d85e3cd7667252 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ec4c05a0c104cdf84d85e3cd7667252" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40853 } } peers { permanent_uuid: "9063ac395df54113ae19a1e57360d327" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 38235 } } peers { permanent_uuid: "5531d5a1827945848cda591f581267f3" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } }
I20260812 03:05:45.572582 28254 master_runner.cc:186] Error getting master registration for 127.5.96.190:40853: OK, instance_id { permanent_uuid: "6ec4c05a0c104cdf84d85e3cd7667252" instance_seqno: 1786503945092402 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:05:45.575627 28254 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:05:45.575778 28246 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ec4c05a0c104cdf84d85e3cd7667252 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 03:05:45.576874 28577 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:05:45.576979 28577 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:05:45.577945 28579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ec4c05a0c104cdf84d85e3cd7667252 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ec4c05a0c104cdf84d85e3cd7667252" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40853 } } peers { permanent_uuid: "9063ac395df54113ae19a1e57360d327" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 38235 } } peers { permanent_uuid: "5531d5a1827945848cda591f581267f3" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } } }
I20260812 03:05:45.578197 28579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ec4c05a0c104cdf84d85e3cd7667252 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 03:05:45.580339 28603 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6ec4c05a0c104cdf84d85e3cd7667252: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:05:45.580417 28603 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:05:45.583721 28119 master_runner.cc:263] No leader master found from master 6ec4c05a0c104cdf84d85e3cd7667252
I20260812 03:05:45.583787 28119 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 03:05:45.589075 28154 tablet.cc:2369] T 00000000000000000000000000000000 P 6ec4c05a0c104cdf84d85e3cd7667252: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:05:45.728225 28555 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:05:45.728446 28555 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ec4c05a0c104cdf84d85e3cd7667252" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40853 } } peers { permanent_uuid: "9063ac395df54113ae19a1e57360d327" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 38235 } } peers { permanent_uuid: "5531d5a1827945848cda591f581267f3" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } }
I20260812 03:05:45.729553 28555 leader_election.cc:288] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ec4c05a0c104cdf84d85e3cd7667252 (127.5.96.190:40853), 5531d5a1827945848cda591f581267f3 (127.5.96.188:40785)
I20260812 03:05:45.730221 28466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9063ac395df54113ae19a1e57360d327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5531d5a1827945848cda591f581267f3" is_pre_election: true
I20260812 03:05:45.730340 28195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9063ac395df54113ae19a1e57360d327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ec4c05a0c104cdf84d85e3cd7667252" is_pre_election: true
I20260812 03:05:45.730543 28466 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5531d5a1827945848cda591f581267f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9063ac395df54113ae19a1e57360d327 in term 0.
I20260812 03:05:45.730644 28195 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6ec4c05a0c104cdf84d85e3cd7667252 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9063ac395df54113ae19a1e57360d327 in term 0.
I20260812 03:05:45.730989 28290 leader_election.cc:302] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5531d5a1827945848cda591f581267f3, 9063ac395df54113ae19a1e57360d327; no voters:
I20260812 03:05:45.731457 28555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:05:45.731552 28555 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:05:45.731602 28555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:05:45.733039 28555 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ec4c05a0c104cdf84d85e3cd7667252" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40853 } } peers { permanent_uuid: "9063ac395df54113ae19a1e57360d327" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 38235 } } peers { permanent_uuid: "5531d5a1827945848cda591f581267f3" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } }
I20260812 03:05:45.733408 28555 leader_election.cc:288] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [CANDIDATE]: Term 1 election: Requested vote from peers 6ec4c05a0c104cdf84d85e3cd7667252 (127.5.96.190:40853), 5531d5a1827945848cda591f581267f3 (127.5.96.188:40785)
I20260812 03:05:45.733752 28195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9063ac395df54113ae19a1e57360d327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ec4c05a0c104cdf84d85e3cd7667252"
I20260812 03:05:45.733857 28195 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ec4c05a0c104cdf84d85e3cd7667252 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:05:45.733896 28466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9063ac395df54113ae19a1e57360d327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5531d5a1827945848cda591f581267f3"
I20260812 03:05:45.734007 28466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5531d5a1827945848cda591f581267f3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:05:45.735164 28195 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6ec4c05a0c104cdf84d85e3cd7667252 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9063ac395df54113ae19a1e57360d327 in term 1.
I20260812 03:05:45.735211 28466 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5531d5a1827945848cda591f581267f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9063ac395df54113ae19a1e57360d327 in term 1.
I20260812 03:05:45.735522 28290 leader_election.cc:302] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5531d5a1827945848cda591f581267f3, 9063ac395df54113ae19a1e57360d327; no voters:
I20260812 03:05:45.735782 28555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:05:45.736068 28555 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [term 1 LEADER]: Becoming Leader. State: Replica: 9063ac395df54113ae19a1e57360d327, State: Running, Role: LEADER
I20260812 03:05:45.736507 28555 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ec4c05a0c104cdf84d85e3cd7667252" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40853 } } peers { permanent_uuid: "9063ac395df54113ae19a1e57360d327" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 38235 } } peers { permanent_uuid: "5531d5a1827945848cda591f581267f3" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } }
W20260812 03:05:45.738273 28425 tablet.cc:2369] T 00000000000000000000000000000000 P 5531d5a1827945848cda591f581267f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:05:45.738957 28605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9063ac395df54113ae19a1e57360d327. Latest consensus state: current_term: 1 leader_uuid: "9063ac395df54113ae19a1e57360d327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ec4c05a0c104cdf84d85e3cd7667252" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40853 } } peers { permanent_uuid: "9063ac395df54113ae19a1e57360d327" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 38235 } } peers { permanent_uuid: "5531d5a1827945848cda591f581267f3" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } } }
I20260812 03:05:45.739120 28605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [sys.catalog]: This master's current role is: LEADER
I20260812 03:05:45.739522 28607 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:05:45.740461 28607 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:05:45.745438 28195 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6ec4c05a0c104cdf84d85e3cd7667252 [term 1 FOLLOWER]: Refusing update from remote peer 9063ac395df54113ae19a1e57360d327: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:05:45.745483 28466 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5531d5a1827945848cda591f581267f3 [term 1 FOLLOWER]: Refusing update from remote peer 9063ac395df54113ae19a1e57360d327: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:05:45.746013 28605 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ec4c05a0c104cdf84d85e3cd7667252" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:05:45.746142 28555 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5531d5a1827945848cda591f581267f3" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:05:45.750057 28528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5531d5a1827945848cda591f581267f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9063ac395df54113ae19a1e57360d327. Latest consensus state: current_term: 1 leader_uuid: "9063ac395df54113ae19a1e57360d327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ec4c05a0c104cdf84d85e3cd7667252" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40853 } } peers { permanent_uuid: "9063ac395df54113ae19a1e57360d327" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 38235 } } peers { permanent_uuid: "5531d5a1827945848cda591f581267f3" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } } }
I20260812 03:05:45.750185 28528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5531d5a1827945848cda591f581267f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:05:45.750231 28579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ec4c05a0c104cdf84d85e3cd7667252 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9063ac395df54113ae19a1e57360d327. Latest consensus state: current_term: 1 leader_uuid: "9063ac395df54113ae19a1e57360d327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ec4c05a0c104cdf84d85e3cd7667252" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40853 } } peers { permanent_uuid: "9063ac395df54113ae19a1e57360d327" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 38235 } } peers { permanent_uuid: "5531d5a1827945848cda591f581267f3" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } } }
I20260812 03:05:45.750332 28579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ec4c05a0c104cdf84d85e3cd7667252 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:05:45.752187 28609 mvcc.cc:205] Tried to move back new op lower bound from 7317520161767124992 to 7317520161739456512. Current Snapshot: MvccSnapshot[applied={T|T < 7317520161767124992}]
I20260812 03:05:45.752413 28618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9063ac395df54113ae19a1e57360d327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ec4c05a0c104cdf84d85e3cd7667252" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40853 } } peers { permanent_uuid: "9063ac395df54113ae19a1e57360d327" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 38235 } } peers { permanent_uuid: "5531d5a1827945848cda591f581267f3" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } } }
I20260812 03:05:45.752516 28618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [sys.catalog]: This master's current role is: LEADER
I20260812 03:05:45.753046 28579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ec4c05a0c104cdf84d85e3cd7667252 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9063ac395df54113ae19a1e57360d327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ec4c05a0c104cdf84d85e3cd7667252" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40853 } } peers { permanent_uuid: "9063ac395df54113ae19a1e57360d327" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 38235 } } peers { permanent_uuid: "5531d5a1827945848cda591f581267f3" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } } }
I20260812 03:05:45.753049 28528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5531d5a1827945848cda591f581267f3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9063ac395df54113ae19a1e57360d327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ec4c05a0c104cdf84d85e3cd7667252" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40853 } } peers { permanent_uuid: "9063ac395df54113ae19a1e57360d327" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 38235 } } peers { permanent_uuid: "5531d5a1827945848cda591f581267f3" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } } }
I20260812 03:05:45.753129 28611 mvcc.cc:205] Tried to move back new op lower bound from 7317520161767124992 to 7317520161739456512. Current Snapshot: MvccSnapshot[applied={T|T < 7317520161767124992}]
I20260812 03:05:45.753156 28528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5531d5a1827945848cda591f581267f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:05:45.753156 28579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ec4c05a0c104cdf84d85e3cd7667252 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:05:45.753607 28555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9063ac395df54113ae19a1e57360d327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ec4c05a0c104cdf84d85e3cd7667252" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 40853 } } peers { permanent_uuid: "9063ac395df54113ae19a1e57360d327" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 38235 } } peers { permanent_uuid: "5531d5a1827945848cda591f581267f3" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 40785 } } }
I20260812 03:05:45.753715 28555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327 [sys.catalog]: This master's current role is: LEADER
I20260812 03:05:45.754722 28607 catalog_manager.cc:1347] Generated new cluster ID: c46fd00fe70b47bf9fd9d38457a55c85
I20260812 03:05:45.754813 28607 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:05:45.787894 28607 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:05:45.789186 28607 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:05:45.796237 28607 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9063ac395df54113ae19a1e57360d327: Generated new TSK 0
I20260812 03:05:45.796975 28607 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:05:46.555120 28547 catalog_manager.cc:1259] Loaded cluster ID: c46fd00fe70b47bf9fd9d38457a55c85
I20260812 03:05:46.555202 28547 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5531d5a1827945848cda591f581267f3: loading cluster ID for follower catalog manager: success
I20260812 03:05:46.556919 28547 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5531d5a1827945848cda591f581267f3: acquiring CA information for follower catalog manager: success
I20260812 03:05:46.558104 28547 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5531d5a1827945848cda591f581267f3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:05:46.581568 28603 catalog_manager.cc:1259] Loaded cluster ID: c46fd00fe70b47bf9fd9d38457a55c85
I20260812 03:05:46.581629 28603 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6ec4c05a0c104cdf84d85e3cd7667252: loading cluster ID for follower catalog manager: success
I20260812 03:05:46.583303 28603 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6ec4c05a0c104cdf84d85e3cd7667252: acquiring CA information for follower catalog manager: success
I20260812 03:05:46.584461 28603 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6ec4c05a0c104cdf84d85e3cd7667252: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:05:46.850422 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 28119
I20260812 03:05:46.863750 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 28254
I20260812 03:05:46.876083 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 28386
2026-08-12T03:05:46Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3219 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:05:46 dist-test-slave-6bbx krb5kdc[28631](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:05:46 dist-test-slave-6bbx krb5kdc[28631](info): set up 2 sockets
Aug 12 03:05:46 dist-test-slave-6bbx krb5kdc[28631](info): commencing operation
krb5kdc: starting...
W20260812 03:05:48.037954 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:05:48 dist-test-slave-6bbx krb5kdc[28631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:05:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:05:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:05:48.147630 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:42681
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:42133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:42681
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:05:48.254088 28659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:05:48.254321 28659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:05:48.254374 28659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:05:48.258234 28659 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:05:48.258284 28659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:05:48.258309 28659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:05:48.258330 28659 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:05:48.258354 28659 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:05:48.263381 28659 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:42133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:42681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:42681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:05:48.264284 28659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:05:48.265177 28659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:05:48.272859 28672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:48.272866 28671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:48.272907 28659 server_base.cc:1034] running on GCE node
W20260812 03:05:48.272858 28675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:48.273733 28659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:05:48.274655 28659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:05:48.275796 28659 hybrid_clock.cc:648] HybridClock initialized: now 1786503948275785 us; error 24 us; skew 500 ppm
Aug 12 03:05:48 dist-test-slave-6bbx krb5kdc[28631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503948, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:05:48.278316 28659 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:05:48.279098 28659 webserver.cc:460] Webserver started at http://127.5.96.190:39091/ using document root <none> and password file <none>
I20260812 03:05:48.279582 28659 fs_manager.cc:359] Metadata directory not provided
I20260812 03:05:48.279647 28659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:05:48.279860 28659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:05:48.281701 28659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "d145794d1be1460f836fc5ab990abbd4"
format_stamp: "Formatted at 2026-08-12 03:05:48 on dist-test-slave-6bbx"
server_key: "47155c8e197687fcd3cc69bc6c464d75"
server_key_iv: "d4e6ad92ebbcadc8a53b3571840f39a2"
server_key_version: "encryptionkey@0"
I20260812 03:05:48.282227 28659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "d145794d1be1460f836fc5ab990abbd4"
format_stamp: "Formatted at 2026-08-12 03:05:48 on dist-test-slave-6bbx"
server_key: "47155c8e197687fcd3cc69bc6c464d75"
server_key_iv: "d4e6ad92ebbcadc8a53b3571840f39a2"
server_key_version: "encryptionkey@0"
I20260812 03:05:48.285589 28659 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 03:05:48.287711 28686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:48.288827 28659 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:05:48.288945 28659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "d145794d1be1460f836fc5ab990abbd4"
format_stamp: "Formatted at 2026-08-12 03:05:48 on dist-test-slave-6bbx"
server_key: "47155c8e197687fcd3cc69bc6c464d75"
server_key_iv: "d4e6ad92ebbcadc8a53b3571840f39a2"
server_key_version: "encryptionkey@0"
I20260812 03:05:48.289033 28659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:05:48.318316 28659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:05:48.321246 28659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:05:48.321415 28659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:05:48.327934 28659 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:42681
I20260812 03:05:48.327970 28769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:42681 every 8 connection(s)
I20260812 03:05:48.328814 28659 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:05:48.331350 28770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:05:48.332859 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 28659
I20260812 03:05:48.332979 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:05:48.333274 5506 external_mini_cluster.cc:1428] Setting key 6d3f76a4335cadd6f9e64396466c675f
I20260812 03:05:48.337189 28770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d145794d1be1460f836fc5ab990abbd4: Bootstrap starting.
I20260812 03:05:48.339280 28770 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d145794d1be1460f836fc5ab990abbd4: Neither blocks nor log segments found. Creating new log.
I20260812 03:05:48.340010 28770 log.cc:824] T 00000000000000000000000000000000 P d145794d1be1460f836fc5ab990abbd4: Log is configured to *not* fsync() on all Append() calls
Aug 12 03:05:48 dist-test-slave-6bbx krb5kdc[28631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:05:48.342031 28770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d145794d1be1460f836fc5ab990abbd4: No bootstrap required, opened a new log
I20260812 03:05:48.345698 28770 raft_consensus.cc:348] T 00000000000000000000000000000000 P d145794d1be1460f836fc5ab990abbd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d145794d1be1460f836fc5ab990abbd4" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42681 } }
I20260812 03:05:48.345911 28770 raft_consensus.cc:374] T 00000000000000000000000000000000 P d145794d1be1460f836fc5ab990abbd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:05:48.345971 28770 raft_consensus.cc:729] T 00000000000000000000000000000000 P d145794d1be1460f836fc5ab990abbd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d145794d1be1460f836fc5ab990abbd4, State: Initialized, Role: FOLLOWER
I20260812 03:05:48.346387 28770 consensus_queue.cc:260] T 00000000000000000000000000000000 P d145794d1be1460f836fc5ab990abbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d145794d1be1460f836fc5ab990abbd4" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42681 } }
I20260812 03:05:48.346473 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:48.334584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51202 (local address 127.5.96.190:42681)
0812 03:05:48.335005 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:48.335023 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:48.335057 (+ 34us) server_negotiation.cc:407] Connection header received
0812 03:05:48.335703 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:48.335725 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:48.336038 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:48.336332 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:48.337117 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:48.338102 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:48.338942 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:48.339435 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:48.339576 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:48.342290 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:48.342295 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:48.342307 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:48.342331 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:48.343827 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:48.344283 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:48.344287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:48.344291 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:48.344406 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:48.344671 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:48.344673 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:48.344674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:48.345020 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:48.345166 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:48.345442 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 03:05:48.345698 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":133,"threads_started":1}
I20260812 03:05:48.346508 28770 raft_consensus.cc:388] T 00000000000000000000000000000000 P d145794d1be1460f836fc5ab990abbd4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:05:48.346560 28770 raft_consensus.cc:482] T 00000000000000000000000000000000 P d145794d1be1460f836fc5ab990abbd4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:05:48.346640 28770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d145794d1be1460f836fc5ab990abbd4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:05:48.347538 28770 raft_consensus.cc:504] T 00000000000000000000000000000000 P d145794d1be1460f836fc5ab990abbd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d145794d1be1460f836fc5ab990abbd4" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42681 } }
I20260812 03:05:48.347915 28770 leader_election.cc:302] T 00000000000000000000000000000000 P d145794d1be1460f836fc5ab990abbd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d145794d1be1460f836fc5ab990abbd4; no voters:
I20260812 03:05:48.348215 28770 leader_election.cc:288] T 00000000000000000000000000000000 P d145794d1be1460f836fc5ab990abbd4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:05:48.348323 28775 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d145794d1be1460f836fc5ab990abbd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:05:48.348587 28775 raft_consensus.cc:686] T 00000000000000000000000000000000 P d145794d1be1460f836fc5ab990abbd4 [term 1 LEADER]: Becoming Leader. State: Replica: d145794d1be1460f836fc5ab990abbd4, State: Running, Role: LEADER
I20260812 03:05:48.348942 28775 consensus_queue.cc:237] T 00000000000000000000000000000000 P d145794d1be1460f836fc5ab990abbd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d145794d1be1460f836fc5ab990abbd4" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42681 } }
I20260812 03:05:48.349315 5506 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:05:48.349478 28770 sys_catalog.cc:564] T 00000000000000000000000000000000 P d145794d1be1460f836fc5ab990abbd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:05:48.350610 28777 sys_catalog.cc:455] T 00000000000000000000000000000000 P d145794d1be1460f836fc5ab990abbd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d145794d1be1460f836fc5ab990abbd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d145794d1be1460f836fc5ab990abbd4" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42681 } } }
I20260812 03:05:48.350610 28776 sys_catalog.cc:455] T 00000000000000000000000000000000 P d145794d1be1460f836fc5ab990abbd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d145794d1be1460f836fc5ab990abbd4. Latest consensus state: current_term: 1 leader_uuid: "d145794d1be1460f836fc5ab990abbd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d145794d1be1460f836fc5ab990abbd4" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42681 } } }
I20260812 03:05:48.350761 28776 sys_catalog.cc:458] T 00000000000000000000000000000000 P d145794d1be1460f836fc5ab990abbd4 [sys.catalog]: This master's current role is: LEADER
I20260812 03:05:48.350761 28777 sys_catalog.cc:458] T 00000000000000000000000000000000 P d145794d1be1460f836fc5ab990abbd4 [sys.catalog]: This master's current role is: LEADER
I20260812 03:05:48.351243 28790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:05:48.353982 28790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:05:48.358904 28790 catalog_manager.cc:1347] Generated new cluster ID: 8fd854ccabad44b18d932b2fed3bc714
I20260812 03:05:48.358968 28790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:05:48.369527 28790 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:05:48.370311 28790 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:05:48.383129 28790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d145794d1be1460f836fc5ab990abbd4: Generated new TSK 0
I20260812 03:05:48.383718 28790 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:05:48.405049 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 28659
2026-08-12T03:05:48Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1533 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:05:48 dist-test-slave-6bbx krb5kdc[28808](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:05:48 dist-test-slave-6bbx krb5kdc[28808](info): set up 2 sockets
Aug 12 03:05:48 dist-test-slave-6bbx krb5kdc[28808](info): commencing operation
krb5kdc: starting...
W20260812 03:05:49.557308 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:05:49 dist-test-slave-6bbx krb5kdc[28808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:05:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:05:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:05:49.668218 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:33641
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:33641
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:05:49.775034 28829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:05:49.775259 28829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:05:49.775296 28829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:05:49.779052 28829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:05:49.779095 28829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:05:49.779111 28829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:05:49.779132 28829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:05:49.779145 28829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:05:49.784138 28829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:33641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:33641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:05:49.784958 28829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:05:49.785802 28829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:05:49.793517 28838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:49.793552 28836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:49.793541 28835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:49.793664 28829 server_base.cc:1034] running on GCE node
I20260812 03:05:49.794293 28829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:05:49.795236 28829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:05:49.796384 28829 hybrid_clock.cc:648] HybridClock initialized: now 1786503949796365 us; error 33 us; skew 500 ppm
Aug 12 03:05:49 dist-test-slave-6bbx krb5kdc[28808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503949, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:05:49.798820 28829 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:05:49.799611 28829 webserver.cc:460] Webserver started at http://127.5.96.190:33439/ using document root <none> and password file <none>
I20260812 03:05:49.800105 28829 fs_manager.cc:359] Metadata directory not provided
I20260812 03:05:49.800172 28829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:05:49.800361 28829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:05:49.802176 28829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "2fa15f69ad6b4d5a897c25d060030d16"
format_stamp: "Formatted at 2026-08-12 03:05:49 on dist-test-slave-6bbx"
server_key: "d2ae9c72ee319e852f913761276a6d56"
server_key_iv: "e5da01a115d3b1f1d7b7adfecde92416"
server_key_version: "encryptionkey@0"
I20260812 03:05:49.802709 28829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "2fa15f69ad6b4d5a897c25d060030d16"
format_stamp: "Formatted at 2026-08-12 03:05:49 on dist-test-slave-6bbx"
server_key: "d2ae9c72ee319e852f913761276a6d56"
server_key_iv: "e5da01a115d3b1f1d7b7adfecde92416"
server_key_version: "encryptionkey@0"
I20260812 03:05:49.806030 28829 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 03:05:49.808207 28853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:49.809279 28829 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:05:49.809387 28829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "2fa15f69ad6b4d5a897c25d060030d16"
format_stamp: "Formatted at 2026-08-12 03:05:49 on dist-test-slave-6bbx"
server_key: "d2ae9c72ee319e852f913761276a6d56"
server_key_iv: "e5da01a115d3b1f1d7b7adfecde92416"
server_key_version: "encryptionkey@0"
I20260812 03:05:49.809471 28829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:05:49.828503 28829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:05:49.830921 28829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:05:49.831082 28829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:05:49.837708 28829 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:33641
I20260812 03:05:49.837952 28943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:33641 every 8 connection(s)
I20260812 03:05:49.838587 28829 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:05:49.841205 28945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:05:49.843045 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 28829
I20260812 03:05:49.843178 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:05:49.843482 5506 external_mini_cluster.cc:1428] Setting key f884b658c41bb4af05bb1d4b0d40477c
I20260812 03:05:49.847517 28945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16: Bootstrap starting.
I20260812 03:05:49.849967 28945 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16: Neither blocks nor log segments found. Creating new log.
I20260812 03:05:49.850811 28945 log.cc:824] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16: Log is configured to *not* fsync() on all Append() calls
Aug 12 03:05:49 dist-test-slave-6bbx krb5kdc[28808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:05:49.852794 28945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16: No bootstrap required, opened a new log
I20260812 03:05:49.856089 28945 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa15f69ad6b4d5a897c25d060030d16" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33641 } }
I20260812 03:05:49.856308 28945 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:05:49.856345 28945 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fa15f69ad6b4d5a897c25d060030d16, State: Initialized, Role: FOLLOWER
I20260812 03:05:49.856750 28945 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa15f69ad6b4d5a897c25d060030d16" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33641 } }
I20260812 03:05:49.856875 28950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:49.844767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39224 (local address 127.5.96.190:33641)
0812 03:05:49.845133 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:49.845148 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:49.845183 (+ 35us) server_negotiation.cc:407] Connection header received
0812 03:05:49.845803 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:49.845820 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:49.846117 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:49.846383 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:49.847244 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:49.848259 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:49.849388 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:49.849816 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:49.849922 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:49.852624 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:49.852628 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:49.852640 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:49.852665 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:49.854162 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:49.854579 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:49.854584 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:49.854588 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:49.854695 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:49.854949 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:49.854953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:49.854954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:49.855326 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:49.855511 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:49.855813 (+ 302us) server_negotiation.cc:299] Negotiation successful
0812 03:05:49.856070 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":106,"threads_started":1}
I20260812 03:05:49.856894 28945 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:05:49.857023 28945 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:05:49.857108 28945 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:05:49.858083 28945 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa15f69ad6b4d5a897c25d060030d16" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33641 } }
I20260812 03:05:49.858394 28945 leader_election.cc:302] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fa15f69ad6b4d5a897c25d060030d16; no voters:
I20260812 03:05:49.858659 28945 leader_election.cc:288] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:05:49.858763 28953 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:05:49.859005 28953 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [term 1 LEADER]: Becoming Leader. State: Replica: 2fa15f69ad6b4d5a897c25d060030d16, State: Running, Role: LEADER
I20260812 03:05:49.859298 28953 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa15f69ad6b4d5a897c25d060030d16" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33641 } }
I20260812 03:05:49.859839 28945 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:05:49.860452 28955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fa15f69ad6b4d5a897c25d060030d16. Latest consensus state: current_term: 1 leader_uuid: "2fa15f69ad6b4d5a897c25d060030d16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa15f69ad6b4d5a897c25d060030d16" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33641 } } }
I20260812 03:05:49.860584 28955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [sys.catalog]: This master's current role is: LEADER
I20260812 03:05:49.860924 28954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2fa15f69ad6b4d5a897c25d060030d16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa15f69ad6b4d5a897c25d060030d16" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33641 } } }
I20260812 03:05:49.861027 28954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [sys.catalog]: This master's current role is: LEADER
I20260812 03:05:49.862289 28965 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:05:49.864943 28965 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:05:49.870116 28965 catalog_manager.cc:1347] Generated new cluster ID: bd50ef7171614e4bb1f091007661d8b5
I20260812 03:05:49.870177 28965 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:05:49.893683 28965 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:05:49.894517 28965 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:05:49.900269 28965 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16: Generated new TSK 0
I20260812 03:05:49.900870 28965 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.96.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
I20260812 03:05:49.933948 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:0
--local_ip_for_outbound_sockets=127.5.96.129
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:33641
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:05:50.070803 28993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:05:50.071019 28993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:05:50.071058 28993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:05:50.074885 28993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:05:50.074935 28993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:05:50.074988 28993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:05:50.079900 28993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:33641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:05:50.080744 28993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:05:50.081638 28993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:05:50.090703 28998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:50.090698 29003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:50.090756 29000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:50.090929 28993 server_base.cc:1034] running on GCE node
I20260812 03:05:50.091353 28993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:05:50.091957 28993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:05:50.093111 28993 hybrid_clock.cc:648] HybridClock initialized: now 1786503950093100 us; error 27 us; skew 500 ppm
Aug 12 03:05:50 dist-test-slave-6bbx krb5kdc[28808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503950, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:05:50.098343 28993 init.cc:377] Logged in from keytab as kudu/127.5.96.129@KRBTEST.COM (short username kudu)
I20260812 03:05:50.099129 28993 webserver.cc:460] Webserver started at http://127.5.96.129:42949/ using document root <none> and password file <none>
I20260812 03:05:50.099640 28993 fs_manager.cc:359] Metadata directory not provided
I20260812 03:05:50.099742 28993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:05:50.099941 28993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:05:50.101795 28993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/data/instance:
uuid: "9d79d5dc5e114111b110c89a2b06456c"
format_stamp: "Formatted at 2026-08-12 03:05:50 on dist-test-slave-6bbx"
server_key: "aeb91d1568259e24245d0b001a286900"
server_key_iv: "092bf30b3d04d7a83cc35ba1d007af63"
server_key_version: "encryptionkey@0"
I20260812 03:05:50.102315 28993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance:
uuid: "9d79d5dc5e114111b110c89a2b06456c"
format_stamp: "Formatted at 2026-08-12 03:05:50 on dist-test-slave-6bbx"
server_key: "aeb91d1568259e24245d0b001a286900"
server_key_iv: "092bf30b3d04d7a83cc35ba1d007af63"
server_key_version: "encryptionkey@0"
I20260812 03:05:50.105710 28993 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 03:05:50.107839 29014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:50.108919 28993 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:05:50.109022 28993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "9d79d5dc5e114111b110c89a2b06456c"
format_stamp: "Formatted at 2026-08-12 03:05:50 on dist-test-slave-6bbx"
server_key: "aeb91d1568259e24245d0b001a286900"
server_key_iv: "092bf30b3d04d7a83cc35ba1d007af63"
server_key_version: "encryptionkey@0"
I20260812 03:05:50.109105 28993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:05:50.127282 28993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:05:50.130116 28993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:05:50.130280 28993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:05:50.130806 28993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:05:50.131552 28993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:05:50.131600 28993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:50.131649 28993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:05:50.131706 28993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:50.139932 28993 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:34127
I20260812 03:05:50.139969 29186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:34127 every 8 connection(s)
I20260812 03:05:50.140736 28993 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
Aug 12 03:05:50 dist-test-slave-6bbx krb5kdc[28808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503950, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:05:50.150401 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 28993
I20260812 03:05:50.150511 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance
I20260812 03:05:50.150751 5506 external_mini_cluster.cc:1428] Setting key 8493373f420fb40e0e77212a3002432a
I20260812 03:05:50.152690 28950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:50.142344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:59923 (local address 127.5.96.190:33641)
0812 03:05:50.142509 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:50.142515 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:50.143467 (+ 952us) server_negotiation.cc:407] Connection header received
0812 03:05:50.144299 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:50.144302 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:50.144347 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:50.144406 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:50.145444 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.145979 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.146994 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.147229 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.147300 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:50.149385 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:50.149388 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:50.149390 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:50.149415 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:50.150721 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:50.151245 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:50.151249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:50.151251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:50.151339 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:50.151642 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:50.151645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:50.151647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:50.151874 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:50.151963 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:50.152471 (+ 508us) server_negotiation.cc:299] Negotiation successful
0812 03:05:50.152569 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:05:50.153617 29192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:50.142706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.129:59923)
0812 03:05:50.143335 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:50.143375 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:50.144116 (+ 741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:50.144544 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:50.144551 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:50.144881 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:05:50.145316 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:50.145329 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.146065 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.146067 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:50.146908 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:50.146917 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.147332 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.147335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:50.147461 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:50.147918 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:05:50.147933 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:05:50.149238 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:05:50.150827 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:50.150833 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:50.150846 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:50.151156 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:50.151421 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:50.151424 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:50.151426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:50.151558 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:50.151980 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:05:50.151986 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:05:50.152234 (+ 248us) client_negotiation.cc:769] Sending connection context
0812 03:05:50.152432 (+ 198us) client_negotiation.cc:240] Negotiation successful
0812 03:05:50.152751 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":112,"threads_started":1}
I20260812 03:05:50.154723 29187 heartbeater.cc:347] Connected to a master server at 127.5.96.190:33641
I20260812 03:05:50.154932 29187 heartbeater.cc:464] Registering TS with master...
I20260812 03:05:50.155359 29187 heartbeater.cc:511] Master 127.5.96.190:33641 requested a full tablet report, sending...
I20260812 03:05:50.156716 28880 ts_manager.cc:194] Registered new tserver with Master: 9d79d5dc5e114111b110c89a2b06456c (127.5.96.129:34127)
I20260812 03:05:50.157831 28880 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.129@KRBTEST.COM'} at 127.5.96.129:59923
WARNING: no policy specified for kudu/127.5.96.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
I20260812 03:05:50.175472 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:0
--local_ip_for_outbound_sockets=127.5.96.130
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:33641
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:05:50.279574 29196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:05:50.279824 29196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:05:50.279882 29196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:05:50.283706 29196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:05:50.283753 29196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:05:50.283825 29196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:05:50.288841 29196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:33641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:05:50.289700 29196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:05:50.290549 29196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:05:50.299185 29209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:50.299226 29206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:50.299237 29212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:50.299477 29196 server_base.cc:1034] running on GCE node
I20260812 03:05:50.299866 29196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:05:50.300410 29196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:05:50.301563 29196 hybrid_clock.cc:648] HybridClock initialized: now 1786503950301554 us; error 28 us; skew 500 ppm
Aug 12 03:05:50 dist-test-slave-6bbx krb5kdc[28808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503950, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:05:50.304128 29196 init.cc:377] Logged in from keytab as kudu/127.5.96.130@KRBTEST.COM (short username kudu)
I20260812 03:05:50.304971 29196 webserver.cc:460] Webserver started at http://127.5.96.130:39461/ using document root <none> and password file <none>
I20260812 03:05:50.305451 29196 fs_manager.cc:359] Metadata directory not provided
I20260812 03:05:50.305514 29196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:05:50.305713 29196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:05:50.307507 29196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/data/instance:
uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0"
format_stamp: "Formatted at 2026-08-12 03:05:50 on dist-test-slave-6bbx"
server_key: "0735851329e2d18cfd3b76501241dd96"
server_key_iv: "762717fba17354a8fd9ba4c4191cc3f8"
server_key_version: "encryptionkey@0"
I20260812 03:05:50.308061 29196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance:
uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0"
format_stamp: "Formatted at 2026-08-12 03:05:50 on dist-test-slave-6bbx"
server_key: "0735851329e2d18cfd3b76501241dd96"
server_key_iv: "762717fba17354a8fd9ba4c4191cc3f8"
server_key_version: "encryptionkey@0"
I20260812 03:05:50.311321 29196 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:05:50.313627 29222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:50.314718 29196 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:05:50.314829 29196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0"
format_stamp: "Formatted at 2026-08-12 03:05:50 on dist-test-slave-6bbx"
server_key: "0735851329e2d18cfd3b76501241dd96"
server_key_iv: "762717fba17354a8fd9ba4c4191cc3f8"
server_key_version: "encryptionkey@0"
I20260812 03:05:50.314915 29196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:05:50.332976 29196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:05:50.335780 29196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:05:50.335934 29196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:05:50.336437 29196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:05:50.337163 29196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:05:50.337213 29196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:50.337262 29196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:05:50.337291 29196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:50.345290 29196 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:32923
I20260812 03:05:50.345314 29398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:32923 every 8 connection(s)
I20260812 03:05:50.346058 29196 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:05:50.350363 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 29196
I20260812 03:05:50.350481 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance
I20260812 03:05:50.350759 5506 external_mini_cluster.cc:1428] Setting key 2d1faf3903c8fba6d7115c7a386bf7bc
Aug 12 03:05:50 dist-test-slave-6bbx krb5kdc[28808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503950, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:05:50.358848 28950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:50.347590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:54693 (local address 127.5.96.190:33641)
0812 03:05:50.347749 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:50.347752 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:50.348672 (+ 920us) server_negotiation.cc:407] Connection header received
0812 03:05:50.349489 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:50.349492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:50.349548 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:50.349620 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:50.350906 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.351522 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.352987 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.353214 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.353290 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:50.355632 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:50.355636 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:50.355639 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:50.355688 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:50.356960 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:50.357430 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:50.357433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:50.357434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:50.357513 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:50.357857 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:50.357861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:50.357862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:50.358031 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:50.358115 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:50.358563 (+ 448us) server_negotiation.cc:299] Negotiation successful
0812 03:05:50.358693 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:05:50.359514 29405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:50.347948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.130:54693)
0812 03:05:50.348541 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:50.348578 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:50.349301 (+ 723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:50.349798 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:50.349805 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:50.350125 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:05:50.350739 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:50.350773 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.351691 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.351694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:50.352874 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:50.352885 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.353350 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.353354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:50.353540 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:50.354019 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:05:50.354039 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:05:50.355475 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:05:50.357070 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:50.357076 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:50.357089 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:50.357337 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:50.357593 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:50.357596 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:50.357599 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:50.357767 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:50.358128 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:05:50.358134 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:05:50.358348 (+ 214us) client_negotiation.cc:769] Sending connection context
0812 03:05:50.358502 (+ 154us) client_negotiation.cc:240] Negotiation successful
0812 03:05:50.358779 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":106,"threads_started":1}
I20260812 03:05:50.360601 29400 heartbeater.cc:347] Connected to a master server at 127.5.96.190:33641
I20260812 03:05:50.360882 29400 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.5.96.131@KRBTEST.COM; defaulting to no policy
I20260812 03:05:50.361397 29400 heartbeater.cc:511] Master 127.5.96.190:33641 requested a full tablet report, sending...
I20260812 03:05:50.362428 28880 ts_manager.cc:194] Registered new tserver with Master: 1a0f8e3c4c9b44ada0371e5a305f2bd0 (127.5.96.130:32923)
I20260812 03:05:50.363065 28880 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.130@KRBTEST.COM'} at 127.5.96.130:54693
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
I20260812 03:05:50.375917 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:0
--local_ip_for_outbound_sockets=127.5.96.131
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:33641
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:05:50.483808 29411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:05:50.484036 29411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:05:50.484076 29411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:05:50.487926 29411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:05:50.487972 29411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:05:50.488032 29411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:05:50.492988 29411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:33641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:05:50.493836 29411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:05:50.494724 29411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:05:50.503914 29426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:50.503953 29422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:50.504154 29420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:50.503996 29411 server_base.cc:1034] running on GCE node
I20260812 03:05:50.504544 29411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:05:50.505131 29411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:05:50.506307 29411 hybrid_clock.cc:648] HybridClock initialized: now 1786503950506283 us; error 41 us; skew 500 ppm
Aug 12 03:05:50 dist-test-slave-6bbx krb5kdc[28808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503950, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:05:50.508857 29411 init.cc:377] Logged in from keytab as kudu/127.5.96.131@KRBTEST.COM (short username kudu)
I20260812 03:05:50.509668 29411 webserver.cc:460] Webserver started at http://127.5.96.131:36517/ using document root <none> and password file <none>
I20260812 03:05:50.510165 29411 fs_manager.cc:359] Metadata directory not provided
I20260812 03:05:50.510231 29411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:05:50.510406 29411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:05:50.512485 29411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/data/instance:
uuid: "96e4c609213140e993ce7f1a08de5ec0"
format_stamp: "Formatted at 2026-08-12 03:05:50 on dist-test-slave-6bbx"
server_key: "2fae027ca5002e691801d719b4490aec"
server_key_iv: "834497e55b4ce0820f7f17c6e9e4de70"
server_key_version: "encryptionkey@0"
I20260812 03:05:50.513070 29411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance:
uuid: "96e4c609213140e993ce7f1a08de5ec0"
format_stamp: "Formatted at 2026-08-12 03:05:50 on dist-test-slave-6bbx"
server_key: "2fae027ca5002e691801d719b4490aec"
server_key_iv: "834497e55b4ce0820f7f17c6e9e4de70"
server_key_version: "encryptionkey@0"
I20260812 03:05:50.516525 29411 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:05:50.518716 29433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:50.519903 29411 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:05:50.520009 29411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "96e4c609213140e993ce7f1a08de5ec0"
format_stamp: "Formatted at 2026-08-12 03:05:50 on dist-test-slave-6bbx"
server_key: "2fae027ca5002e691801d719b4490aec"
server_key_iv: "834497e55b4ce0820f7f17c6e9e4de70"
server_key_version: "encryptionkey@0"
I20260812 03:05:50.520094 29411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:05:50.546242 29411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:05:50.548825 29411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:05:50.548988 29411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:05:50.549527 29411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:05:50.550302 29411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:05:50.550350 29411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:50.550401 29411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:05:50.550429 29411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:50.558607 29411 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:45251
I20260812 03:05:50.558621 29621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:45251 every 8 connection(s)
I20260812 03:05:50.559399 29411 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:05:50.561419 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 29411
I20260812 03:05:50.561514 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance
I20260812 03:05:50.561770 5506 external_mini_cluster.cc:1428] Setting key 058428568f2a0443322bfd339e6320c6
Aug 12 03:05:50 dist-test-slave-6bbx krb5kdc[28808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503950, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:05:50.571368 28950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:50.561081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:41933 (local address 127.5.96.190:33641)
0812 03:05:50.561206 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:50.561211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:50.562387 (+ 1176us) server_negotiation.cc:407] Connection header received
0812 03:05:50.563283 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:50.563286 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:50.563342 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:50.563411 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:50.564588 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.565102 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.566178 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.566419 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.566509 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:50.568736 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:50.568739 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:50.568741 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:50.568765 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:50.569837 (+ 1072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:50.570252 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:50.570255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:50.570256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:50.570321 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:50.570559 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:50.570560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:50.570561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:50.570727 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:50.570797 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:50.571165 (+ 368us) server_negotiation.cc:299] Negotiation successful
0812 03:05:50.571250 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:05:50.572139 29624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:50.561478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.131:41933)
0812 03:05:50.562256 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:50.562296 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:50.563085 (+ 789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:50.563584 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:50.563590 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:50.564019 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:05:50.564458 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:50.564471 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.565214 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.565218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:50.566065 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:50.566075 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.566597 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.566601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:50.566746 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:50.567175 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:05:50.567191 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:05:50.568591 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:05:50.569940 (+ 1349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:50.569944 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:50.569954 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:50.570184 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:50.570380 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:50.570382 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:50.570383 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:50.570500 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:50.570795 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:05:50.570800 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:05:50.570997 (+ 197us) client_negotiation.cc:769] Sending connection context
0812 03:05:50.571134 (+ 137us) client_negotiation.cc:240] Negotiation successful
0812 03:05:50.571364 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":114,"threads_started":1}
I20260812 03:05:50.573144 29622 heartbeater.cc:347] Connected to a master server at 127.5.96.190:33641
I20260812 03:05:50.573343 29622 heartbeater.cc:464] Registering TS with master...
I20260812 03:05:50.573776 29622 heartbeater.cc:511] Master 127.5.96.190:33641 requested a full tablet report, sending...
I20260812 03:05:50.574728 28880 ts_manager.cc:194] Registered new tserver with Master: 96e4c609213140e993ce7f1a08de5ec0 (127.5.96.131:45251)
I20260812 03:05:50.575289 28880 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.131@KRBTEST.COM'} at 127.5.96.131:41933
I20260812 03:05:50.575407 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:05:50.583623 28950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:50.577539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39230 (local address 127.5.96.190:33641)
0812 03:05:50.577674 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:50.577678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:50.577834 (+ 156us) server_negotiation.cc:407] Connection header received
0812 03:05:50.577903 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:50.577905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:50.577939 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:50.577998 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:50.578578 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.579043 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.579803 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.580002 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.580072 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:50.581290 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:50.581293 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:50.581295 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:50.581316 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:50.582359 (+ 1043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:50.582694 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:50.582696 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:50.582697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:50.582766 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:50.582944 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:50.582945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:50.582946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:50.583123 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:50.583218 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:50.583389 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 03:05:50.583490 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 03:05:50.586040 28880 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39230:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:05:50.588969 28880 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:05:50.602325 29648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:50.597017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:32923 (local address 127.0.0.1:53298)
0812 03:05:50.597231 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:50.597244 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:50.597324 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:50.598201 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:50.598204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:50.598219 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:05:50.598310 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:50.598316 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.599815 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.599819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:50.600829 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:50.600839 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.601896 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.601899 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:50.601987 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:50.602008 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 03:05:50.602055 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 03:05:50.602139 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":39,"thread_start_us":91,"threads_started":1}
I20260812 03:05:50.602428 29652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:50.597805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45456 (local address 127.5.96.131:45251)
0812 03:05:50.598162 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:50.598174 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:50.598193 (+ 19us) server_negotiation.cc:407] Connection header received
0812 03:05:50.598258 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:50.598264 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:50.598393 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:50.598517 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:05:50.598723 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.599418 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.601113 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.601891 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.601986 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:50.602131 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:50.602225 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 03:05:50.602308 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":85,"threads_started":1}
I20260812 03:05:50.602432 29649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:50.597154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:45251 (local address 127.0.0.1:45456)
0812 03:05:50.597358 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:50.597370 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:50.597453 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:50.598519 (+ 1066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:50.598522 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:50.598538 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:05:50.598626 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:50.598633 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.599541 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.599543 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:50.601008 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:50.601016 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.601997 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.602001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:50.602107 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:50.602129 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:05:50.602200 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 03:05:50.602265 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":29,"thread_start_us":77,"threads_started":1}
I20260812 03:05:50.602656 29647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:50.596637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45006 (local address 127.5.96.129:34127)
0812 03:05:50.597418 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:50.597428 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:50.597448 (+ 20us) server_negotiation.cc:407] Connection header received
0812 03:05:50.597508 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:50.597512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:50.597697 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:50.598191 (+ 494us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:05:50.598382 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.599536 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.601066 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.601619 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.601718 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:50.601875 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:50.602226 (+ 351us) server_negotiation.cc:299] Negotiation successful
0812 03:05:50.602333 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":129,"threads_started":1}
I20260812 03:05:50.602828 29644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:50.596523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:34127 (local address 127.0.0.1:45006)
0812 03:05:50.596841 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:50.596887 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:50.597006 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:50.597869 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:50.597876 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:50.597899 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:05:50.598023 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:50.598029 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.599697 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.599700 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:50.600923 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:50.600934 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.601738 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.601742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:50.601856 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:50.601892 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 03:05:50.601991 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 03:05:50.602118 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":23,"thread_start_us":68,"threads_started":1}
I20260812 03:05:50.603318 29651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:50.597430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53298 (local address 127.5.96.130:32923)
0812 03:05:50.597775 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:50.597786 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:50.597809 (+ 23us) server_negotiation.cc:407] Connection header received
0812 03:05:50.597877 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:50.597882 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:50.598060 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:50.598452 (+ 392us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:05:50.598606 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.599692 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.600956 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.601751 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.602916 (+ 1165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:50.603049 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:50.603093 (+ 44us) server_negotiation.cc:299] Negotiation successful
0812 03:05:50.603172 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":121,"threads_started":1}
I20260812 03:05:50.605571 29291 tablet_service.cc:1467] Processing CreateTablet for tablet e44de8fccc79450a81c4e1b43116f761 (DEFAULT_TABLE table=test-table [id=abbfc5196fed45809eba4e55ec8b0040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:05:50.605513 29511 tablet_service.cc:1467] Processing CreateTablet for tablet e44de8fccc79450a81c4e1b43116f761 (DEFAULT_TABLE table=test-table [id=abbfc5196fed45809eba4e55ec8b0040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:05:50.605517 29087 tablet_service.cc:1467] Processing CreateTablet for tablet e44de8fccc79450a81c4e1b43116f761 (DEFAULT_TABLE table=test-table [id=abbfc5196fed45809eba4e55ec8b0040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:05:50.606457 29291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e44de8fccc79450a81c4e1b43116f761. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:05:50.606498 29087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e44de8fccc79450a81c4e1b43116f761. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:05:50.606504 29511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e44de8fccc79450a81c4e1b43116f761. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:05:50.612713 29653 tablet_bootstrap.cc:492] T e44de8fccc79450a81c4e1b43116f761 P 1a0f8e3c4c9b44ada0371e5a305f2bd0: Bootstrap starting.
I20260812 03:05:50.613793 29655 tablet_bootstrap.cc:492] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c: Bootstrap starting.
I20260812 03:05:50.614444 29654 tablet_bootstrap.cc:492] T e44de8fccc79450a81c4e1b43116f761 P 96e4c609213140e993ce7f1a08de5ec0: Bootstrap starting.
I20260812 03:05:50.615469 29653 tablet_bootstrap.cc:654] T e44de8fccc79450a81c4e1b43116f761 P 1a0f8e3c4c9b44ada0371e5a305f2bd0: Neither blocks nor log segments found. Creating new log.
I20260812 03:05:50.615906 29655 tablet_bootstrap.cc:654] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c: Neither blocks nor log segments found. Creating new log.
I20260812 03:05:50.616142 29654 tablet_bootstrap.cc:654] T e44de8fccc79450a81c4e1b43116f761 P 96e4c609213140e993ce7f1a08de5ec0: Neither blocks nor log segments found. Creating new log.
I20260812 03:05:50.616392 29653 log.cc:824] T e44de8fccc79450a81c4e1b43116f761 P 1a0f8e3c4c9b44ada0371e5a305f2bd0: Log is configured to *not* fsync() on all Append() calls
I20260812 03:05:50.617060 29655 log.cc:824] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c: Log is configured to *not* fsync() on all Append() calls
I20260812 03:05:50.617247 29654 log.cc:824] T e44de8fccc79450a81c4e1b43116f761 P 96e4c609213140e993ce7f1a08de5ec0: Log is configured to *not* fsync() on all Append() calls
I20260812 03:05:50.618539 29653 tablet_bootstrap.cc:492] T e44de8fccc79450a81c4e1b43116f761 P 1a0f8e3c4c9b44ada0371e5a305f2bd0: No bootstrap required, opened a new log
I20260812 03:05:50.618727 29653 ts_tablet_manager.cc:1397] T e44de8fccc79450a81c4e1b43116f761 P 1a0f8e3c4c9b44ada0371e5a305f2bd0: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260812 03:05:50.618921 29655 tablet_bootstrap.cc:492] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c: No bootstrap required, opened a new log
I20260812 03:05:50.619093 29655 ts_tablet_manager.cc:1397] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 03:05:50.619414 29654 tablet_bootstrap.cc:492] T e44de8fccc79450a81c4e1b43116f761 P 96e4c609213140e993ce7f1a08de5ec0: No bootstrap required, opened a new log
I20260812 03:05:50.619587 29654 ts_tablet_manager.cc:1397] T e44de8fccc79450a81c4e1b43116f761 P 96e4c609213140e993ce7f1a08de5ec0: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 03:05:50.621796 29655 raft_consensus.cc:348] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } } peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } }
I20260812 03:05:50.621995 29655 raft_consensus.cc:374] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:05:50.622047 29655 raft_consensus.cc:729] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d79d5dc5e114111b110c89a2b06456c, State: Initialized, Role: FOLLOWER
I20260812 03:05:50.622450 29655 consensus_queue.cc:260] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } } peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } }
I20260812 03:05:50.622470 29653 raft_consensus.cc:348] T e44de8fccc79450a81c4e1b43116f761 P 1a0f8e3c4c9b44ada0371e5a305f2bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } } peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } }
I20260812 03:05:50.622735 29653 raft_consensus.cc:374] T e44de8fccc79450a81c4e1b43116f761 P 1a0f8e3c4c9b44ada0371e5a305f2bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:05:50.622804 29653 raft_consensus.cc:729] T e44de8fccc79450a81c4e1b43116f761 P 1a0f8e3c4c9b44ada0371e5a305f2bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a0f8e3c4c9b44ada0371e5a305f2bd0, State: Initialized, Role: FOLLOWER
I20260812 03:05:50.623147 29187 heartbeater.cc:503] Master 127.5.96.190:33641 was elected leader, sending a full tablet report...
I20260812 03:05:50.623164 29654 raft_consensus.cc:348] T e44de8fccc79450a81c4e1b43116f761 P 96e4c609213140e993ce7f1a08de5ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } } peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } }
I20260812 03:05:50.623404 29654 raft_consensus.cc:374] T e44de8fccc79450a81c4e1b43116f761 P 96e4c609213140e993ce7f1a08de5ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:05:50.623319 29653 consensus_queue.cc:260] T e44de8fccc79450a81c4e1b43116f761 P 1a0f8e3c4c9b44ada0371e5a305f2bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } } peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } }
I20260812 03:05:50.623441 29655 ts_tablet_manager.cc:1428] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260812 03:05:50.623448 29654 raft_consensus.cc:729] T e44de8fccc79450a81c4e1b43116f761 P 96e4c609213140e993ce7f1a08de5ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96e4c609213140e993ce7f1a08de5ec0, State: Initialized, Role: FOLLOWER
I20260812 03:05:50.623947 29654 consensus_queue.cc:260] T e44de8fccc79450a81c4e1b43116f761 P 96e4c609213140e993ce7f1a08de5ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } } peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } }
I20260812 03:05:50.624276 29653 ts_tablet_manager.cc:1428] T e44de8fccc79450a81c4e1b43116f761 P 1a0f8e3c4c9b44ada0371e5a305f2bd0: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260812 03:05:50.624593 29622 heartbeater.cc:503] Master 127.5.96.190:33641 was elected leader, sending a full tablet report...
I20260812 03:05:50.624656 29400 heartbeater.cc:503] Master 127.5.96.190:33641 was elected leader, sending a full tablet report...
I20260812 03:05:50.625723 29654 ts_tablet_manager.cc:1428] T e44de8fccc79450a81c4e1b43116f761 P 96e4c609213140e993ce7f1a08de5ec0: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260812 03:05:50.641705 29191 tablet.cc:2369] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:05:50.764930 29664 raft_consensus.cc:482] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:05:50.765125 29664 raft_consensus.cc:504] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } } peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } }
I20260812 03:05:50.766247 29664 leader_election.cc:288] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96e4c609213140e993ce7f1a08de5ec0 (127.5.96.131:45251), 1a0f8e3c4c9b44ada0371e5a305f2bd0 (127.5.96.130:32923)
I20260812 03:05:50.769858 29652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:50.766571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:60845 (local address 127.5.96.131:45251)
0812 03:05:50.766889 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:50.766893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:50.766907 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:05:50.766958 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:50.766962 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:50.767003 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:50.767063 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:05:50.767342 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.767935 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.768977 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.769359 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.769462 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:50.769514 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:50.769679 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 03:05:50.769754 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235}
I20260812 03:05:50.769872 29668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:50.766449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:45251 (local address 127.5.96.129:60845)
0812 03:05:50.766750 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:50.766766 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:50.766865 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:50.767090 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:50.767093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:50.767110 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:05:50.767219 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:50.767232 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.768044 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.768047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:50.768852 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:50.768861 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.769468 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.769471 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:50.769565 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:50.769584 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 03:05:50.769649 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 03:05:50.769719 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":73,"threads_started":1}
I20260812 03:05:50.769938 29669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:50.766497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:32923 (local address 127.5.96.129:42323)
0812 03:05:50.766916 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:50.766929 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:50.767034 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:50.767266 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:50.767269 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:50.767285 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:05:50.767401 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:50.767406 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.768140 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.768143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:50.768896 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:50.768905 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.769521 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.769525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:50.769640 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:50.769659 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 03:05:50.769723 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 03:05:50.769791 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":157,"threads_started":1}
I20260812 03:05:50.769922 29651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:50.766618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:42323 (local address 127.5.96.130:32923)
0812 03:05:50.766901 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:50.766905 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:50.767002 (+ 97us) server_negotiation.cc:407] Connection header received
0812 03:05:50.767138 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:50.767141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:50.767182 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:50.767245 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:05:50.767506 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.768027 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.769017 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.769410 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.769513 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:50.769562 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:50.769752 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 03:05:50.769813 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 03:05:50.770462 29552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e44de8fccc79450a81c4e1b43116f761" candidate_uuid: "9d79d5dc5e114111b110c89a2b06456c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96e4c609213140e993ce7f1a08de5ec0" is_pre_election: true
I20260812 03:05:50.770499 29330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e44de8fccc79450a81c4e1b43116f761" candidate_uuid: "9d79d5dc5e114111b110c89a2b06456c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" is_pre_election: true
I20260812 03:05:50.770741 29552 raft_consensus.cc:2454] T e44de8fccc79450a81c4e1b43116f761 P 96e4c609213140e993ce7f1a08de5ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d79d5dc5e114111b110c89a2b06456c in term 0.
I20260812 03:05:50.770750 29330 raft_consensus.cc:2454] T e44de8fccc79450a81c4e1b43116f761 P 1a0f8e3c4c9b44ada0371e5a305f2bd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d79d5dc5e114111b110c89a2b06456c in term 0.
I20260812 03:05:50.771160 29018 leader_election.cc:302] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96e4c609213140e993ce7f1a08de5ec0, 9d79d5dc5e114111b110c89a2b06456c; no voters:
I20260812 03:05:50.771405 29664 raft_consensus.cc:2781] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:05:50.771492 29664 raft_consensus.cc:482] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:05:50.771541 29664 raft_consensus.cc:3037] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:05:50.772555 29664 raft_consensus.cc:504] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } } peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } }
I20260812 03:05:50.772861 29664 leader_election.cc:288] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 1 election: Requested vote from peers 96e4c609213140e993ce7f1a08de5ec0 (127.5.96.131:45251), 1a0f8e3c4c9b44ada0371e5a305f2bd0 (127.5.96.130:32923)
I20260812 03:05:50.773193 29552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e44de8fccc79450a81c4e1b43116f761" candidate_uuid: "9d79d5dc5e114111b110c89a2b06456c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96e4c609213140e993ce7f1a08de5ec0"
I20260812 03:05:50.773193 29330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e44de8fccc79450a81c4e1b43116f761" candidate_uuid: "9d79d5dc5e114111b110c89a2b06456c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0"
I20260812 03:05:50.773313 29330 raft_consensus.cc:3037] T e44de8fccc79450a81c4e1b43116f761 P 1a0f8e3c4c9b44ada0371e5a305f2bd0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:05:50.773313 29552 raft_consensus.cc:3037] T e44de8fccc79450a81c4e1b43116f761 P 96e4c609213140e993ce7f1a08de5ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:05:50.774514 29330 raft_consensus.cc:2454] T e44de8fccc79450a81c4e1b43116f761 P 1a0f8e3c4c9b44ada0371e5a305f2bd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d79d5dc5e114111b110c89a2b06456c in term 1.
I20260812 03:05:50.774514 29552 raft_consensus.cc:2454] T e44de8fccc79450a81c4e1b43116f761 P 96e4c609213140e993ce7f1a08de5ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d79d5dc5e114111b110c89a2b06456c in term 1.
I20260812 03:05:50.774786 29015 leader_election.cc:302] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1a0f8e3c4c9b44ada0371e5a305f2bd0, 96e4c609213140e993ce7f1a08de5ec0, 9d79d5dc5e114111b110c89a2b06456c; no voters:
I20260812 03:05:50.774950 29664 raft_consensus.cc:2781] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:05:50.775151 29664 raft_consensus.cc:686] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c [term 1 LEADER]: Becoming Leader. State: Replica: 9d79d5dc5e114111b110c89a2b06456c, State: Running, Role: LEADER
I20260812 03:05:50.775473 29664 consensus_queue.cc:237] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } } peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } }
I20260812 03:05:50.778496 28877 catalog_manager.cc:5495] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d79d5dc5e114111b110c89a2b06456c (127.5.96.129). New cstate: current_term: 1 leader_uuid: "9d79d5dc5e114111b110c89a2b06456c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } health_report { overall_health: UNKNOWN } } }
W20260812 03:05:50.810321 29623 tablet.cc:2369] T e44de8fccc79450a81c4e1b43116f761 P 96e4c609213140e993ce7f1a08de5ec0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:05:50.833623 29647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:50.830137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45010 (local address 127.5.96.129:34127)
0812 03:05:50.830271 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:50.830275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:50.830289 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:05:50.830399 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:50.830401 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:50.830456 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:50.830547 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:05:50.830805 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.831306 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.832270 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:50.832484 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:50.832576 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:50.832853 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:05:50.833256 (+ 403us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:05:50.833360 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:50.833436 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 03:05:50.833494 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:05:50.839357 29552 raft_consensus.cc:1261] T e44de8fccc79450a81c4e1b43116f761 P 96e4c609213140e993ce7f1a08de5ec0 [term 1 FOLLOWER]: Refusing update from remote peer 9d79d5dc5e114111b110c89a2b06456c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:05:50.839358 29330 raft_consensus.cc:1261] T e44de8fccc79450a81c4e1b43116f761 P 1a0f8e3c4c9b44ada0371e5a305f2bd0 [term 1 FOLLOWER]: Refusing update from remote peer 9d79d5dc5e114111b110c89a2b06456c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:05:50.840127 29671 consensus_queue.cc:1035] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c [LEADER]: Connected to new peer: Peer: permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:05:50.840304 29664 consensus_queue.cc:1035] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 03:05:50.847033 29402 tablet.cc:2369] T e44de8fccc79450a81c4e1b43116f761 P 1a0f8e3c4c9b44ada0371e5a305f2bd0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:05:50.847500 29676 mvcc.cc:205] Tried to move back new op lower bound from 7317520182630924288 to 7317520182379180032. Current Snapshot: MvccSnapshot[applied={T|T < 7317520182630924288}]
I20260812 03:05:50.853400 29678 mvcc.cc:205] Tried to move back new op lower bound from 7317520182630924288 to 7317520182379180032. Current Snapshot: MvccSnapshot[applied={T|T < 7317520182630924288}]
I20260812 03:05:50.857252 28877 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:05:50.857380 28877 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:05:50.859449 28877 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16: Sending DeleteTablet for 3 replicas of tablet e44de8fccc79450a81c4e1b43116f761
I20260812 03:05:50.860205 29511 tablet_service.cc:1514] Processing DeleteTablet for tablet e44de8fccc79450a81c4e1b43116f761 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:05:50 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:45456
I20260812 03:05:50.860205 29087 tablet_service.cc:1514] Processing DeleteTablet for tablet e44de8fccc79450a81c4e1b43116f761 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:05:50 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:45006
I20260812 03:05:50.860235 29291 tablet_service.cc:1514] Processing DeleteTablet for tablet e44de8fccc79450a81c4e1b43116f761 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:05:50 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:53298
I20260812 03:05:50.860693 29699 tablet_replica.cc:330] T e44de8fccc79450a81c4e1b43116f761 P 1a0f8e3c4c9b44ada0371e5a305f2bd0: stopping tablet replica
I20260812 03:05:50.860814 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 28993
I20260812 03:05:50.860971 29699 raft_consensus.cc:2229] T e44de8fccc79450a81c4e1b43116f761 P 1a0f8e3c4c9b44ada0371e5a305f2bd0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:05:50.861240 29699 raft_consensus.cc:2258] T e44de8fccc79450a81c4e1b43116f761 P 1a0f8e3c4c9b44ada0371e5a305f2bd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:05:50.868099 29699 ts_tablet_manager.cc:1905] T e44de8fccc79450a81c4e1b43116f761 P 1a0f8e3c4c9b44ada0371e5a305f2bd0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:05:50.869547 29698 tablet_replica.cc:330] T e44de8fccc79450a81c4e1b43116f761 P 96e4c609213140e993ce7f1a08de5ec0: stopping tablet replica
I20260812 03:05:50.869813 29698 raft_consensus.cc:2229] T e44de8fccc79450a81c4e1b43116f761 P 96e4c609213140e993ce7f1a08de5ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:05:50.870085 29698 raft_consensus.cc:2258] T e44de8fccc79450a81c4e1b43116f761 P 96e4c609213140e993ce7f1a08de5ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:05:50.870620 29699 ts_tablet_manager.cc:1918] T e44de8fccc79450a81c4e1b43116f761 P 1a0f8e3c4c9b44ada0371e5a305f2bd0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:05:50.870719 29699 log.cc:1196] T e44de8fccc79450a81c4e1b43116f761 P 1a0f8e3c4c9b44ada0371e5a305f2bd0: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/wal/wals/e44de8fccc79450a81c4e1b43116f761
I20260812 03:05:50.870970 29699 ts_tablet_manager.cc:1939] T e44de8fccc79450a81c4e1b43116f761 P 1a0f8e3c4c9b44ada0371e5a305f2bd0: Deleting consensus metadata
I20260812 03:05:50.872025 28855 catalog_manager.cc:4841] TS 1a0f8e3c4c9b44ada0371e5a305f2bd0 (127.5.96.130:32923): tablet e44de8fccc79450a81c4e1b43116f761 (table test-table [id=abbfc5196fed45809eba4e55ec8b0040]) successfully deleted
I20260812 03:05:50.876559 29698 ts_tablet_manager.cc:1905] T e44de8fccc79450a81c4e1b43116f761 P 96e4c609213140e993ce7f1a08de5ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:05:50.878652 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 29196
W20260812 03:05:50.878636 28858 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.96.129:34127 (error 108)
I20260812 03:05:50.881776 29698 ts_tablet_manager.cc:1918] T e44de8fccc79450a81c4e1b43116f761 P 96e4c609213140e993ce7f1a08de5ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:05:50.881877 29698 log.cc:1196] T e44de8fccc79450a81c4e1b43116f761 P 96e4c609213140e993ce7f1a08de5ec0: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/wal/wals/e44de8fccc79450a81c4e1b43116f761
I20260812 03:05:50.882218 29698 ts_tablet_manager.cc:1939] T e44de8fccc79450a81c4e1b43116f761 P 96e4c609213140e993ce7f1a08de5ec0: Deleting consensus metadata
I20260812 03:05:50.882822 29649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:50.882493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:34127 (local address 127.0.0.1:45016)
0812 03:05:50.882651 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:50.882735 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.129:34127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260812 03:05:50.883002 28858 catalog_manager.cc:4584] TS 9d79d5dc5e114111b110c89a2b06456c (127.5.96.129:34127): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e44de8fccc79450a81c4e1b43116f761: Network error: Client connection negotiation failed: client connection to 127.5.96.129:34127: connect: Connection refused (error 111)
I20260812 03:05:50.883289 28858 catalog_manager.cc:4841] TS 96e4c609213140e993ce7f1a08de5ec0 (127.5.96.131:45251): tablet e44de8fccc79450a81c4e1b43116f761 (table test-table [id=abbfc5196fed45809eba4e55ec8b0040]) successfully deleted
I20260812 03:05:50.887225 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 29411
I20260812 03:05:50.899488 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 28829
I20260812 03:05:50.910550 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:33641
--webserver_interface=127.5.96.190
--webserver_port=33439
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:33641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:05:51.017167 29703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:05:51.017390 29703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:05:51.017427 29703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:05:51.021230 29703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:05:51.021274 29703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:05:51.021291 29703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:05:51.021309 29703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:05:51.021325 29703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:05:51.026244 29703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:33641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:33641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=33439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:05:51.027060 29703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:05:51.027954 29703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:05:51.041464 29713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:51.041455 29712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:51.041565 29703 server_base.cc:1034] running on GCE node
W20260812 03:05:51.041455 29717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:51.042299 29703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:05:51.043322 29703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:05:51.044462 29703 hybrid_clock.cc:648] HybridClock initialized: now 1786503951044437 us; error 37 us; skew 500 ppm
Aug 12 03:05:51 dist-test-slave-6bbx krb5kdc[28808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503951, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:05:51.047070 29703 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:05:51.047953 29703 webserver.cc:460] Webserver started at http://127.5.96.190:33439/ using document root <none> and password file <none>
I20260812 03:05:51.048470 29703 fs_manager.cc:359] Metadata directory not provided
I20260812 03:05:51.048549 29703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:05:51.052177 29703 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:05:51.054118 29726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:51.055231 29703 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:05:51.055346 29703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "2fa15f69ad6b4d5a897c25d060030d16"
format_stamp: "Formatted at 2026-08-12 03:05:49 on dist-test-slave-6bbx"
server_key: "d2ae9c72ee319e852f913761276a6d56"
server_key_iv: "e5da01a115d3b1f1d7b7adfecde92416"
server_key_version: "encryptionkey@0"
I20260812 03:05:51.055932 29703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:05:51.069279 29703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:05:51.072758 29703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:05:51.072913 29703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:05:51.080806 29703 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:33641
I20260812 03:05:51.081156 29811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:33641 every 8 connection(s)
I20260812 03:05:51.082055 29703 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:05:51.086385 29814 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:05:51.086424 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 29703
I20260812 03:05:51.086820 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:34127
--local_ip_for_outbound_sockets=127.5.96.129
--tserver_master_addrs=127.5.96.190:33641
--webserver_port=42949
--webserver_interface=127.5.96.129
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:05:51.088110 29814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16: Bootstrap starting.
I20260812 03:05:51.104678 29814 log.cc:824] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16: Log is configured to *not* fsync() on all Append() calls
I20260812 03:05:51.112953 29814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:05:51.113252 29814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16: Bootstrap complete.
I20260812 03:05:51.117116 29814 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa15f69ad6b4d5a897c25d060030d16" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33641 } }
I20260812 03:05:51.117487 29814 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fa15f69ad6b4d5a897c25d060030d16, State: Initialized, Role: FOLLOWER
I20260812 03:05:51.117930 29814 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa15f69ad6b4d5a897c25d060030d16" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33641 } }
I20260812 03:05:51.118050 29814 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:05:51.118119 29814 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:05:51.118214 29814 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:05:51.120028 29814 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa15f69ad6b4d5a897c25d060030d16" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33641 } }
I20260812 03:05:51.120333 29814 leader_election.cc:302] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fa15f69ad6b4d5a897c25d060030d16; no voters:
I20260812 03:05:51.120599 29814 leader_election.cc:288] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 03:05:51.120710 29823 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:05:51.120949 29823 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [term 2 LEADER]: Becoming Leader. State: Replica: 2fa15f69ad6b4d5a897c25d060030d16, State: Running, Role: LEADER
I20260812 03:05:51.121244 29823 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa15f69ad6b4d5a897c25d060030d16" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33641 } }
I20260812 03:05:51.121717 29814 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:05:51.122690 29825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fa15f69ad6b4d5a897c25d060030d16. Latest consensus state: current_term: 2 leader_uuid: "2fa15f69ad6b4d5a897c25d060030d16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa15f69ad6b4d5a897c25d060030d16" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33641 } } }
I20260812 03:05:51.122818 29825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [sys.catalog]: This master's current role is: LEADER
I20260812 03:05:51.123064 29824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2fa15f69ad6b4d5a897c25d060030d16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa15f69ad6b4d5a897c25d060030d16" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33641 } } }
I20260812 03:05:51.123155 29824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [sys.catalog]: This master's current role is: LEADER
I20260812 03:05:51.123281 29835 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:05:51.127863 29835 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:05:51.128504 29835 catalog_manager.cc:1259] Loaded cluster ID: bd50ef7171614e4bb1f091007661d8b5
I20260812 03:05:51.128562 29835 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:05:51.129909 29835 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 03:05:51.134287 29835 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 03:05:51.139128 29835 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:05:51.139837 29835 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16: Loaded TSK: 0
I20260812 03:05:51.141007 29835 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:05:51.201501 29818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:05:51.201721 29818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:05:51.201766 29818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:05:51.205677 29818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:05:51.205725 29818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:05:51.205786 29818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:05:51.210703 29818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:34127
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=42949
--tserver_master_addrs=127.5.96.190:33641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:05:51.211606 29818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:05:51.212563 29818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:05:51.222662 29864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:51.222632 29862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:51.223011 29870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:51.223130 29818 server_base.cc:1034] running on GCE node
I20260812 03:05:51.223721 29818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:05:51.224486 29818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:05:51.225694 29818 hybrid_clock.cc:648] HybridClock initialized: now 1786503951225664 us; error 37 us; skew 500 ppm
Aug 12 03:05:51 dist-test-slave-6bbx krb5kdc[28808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503951, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:05:51.228924 29818 init.cc:377] Logged in from keytab as kudu/127.5.96.129@KRBTEST.COM (short username kudu)
I20260812 03:05:51.229784 29818 webserver.cc:460] Webserver started at http://127.5.96.129:42949/ using document root <none> and password file <none>
I20260812 03:05:51.230302 29818 fs_manager.cc:359] Metadata directory not provided
I20260812 03:05:51.230376 29818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:05:51.234184 29818 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:05:51.236372 29878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:51.237648 29818 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:05:51.237771 29818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "9d79d5dc5e114111b110c89a2b06456c"
format_stamp: "Formatted at 2026-08-12 03:05:50 on dist-test-slave-6bbx"
server_key: "aeb91d1568259e24245d0b001a286900"
server_key_iv: "092bf30b3d04d7a83cc35ba1d007af63"
server_key_version: "encryptionkey@0"
I20260812 03:05:51.238360 29818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:05:51.264712 29818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:05:51.267759 29818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:05:51.267942 29818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:05:51.268525 29818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:05:51.269558 29891 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 03:05:51.273578 29818 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 03:05:51.273650 29818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260812 03:05:51.273702 29818 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 03:05:51.275312 29818 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 03:05:51.275357 29818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260812 03:05:51.275609 29891 tablet_bootstrap.cc:492] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c: Bootstrap starting.
I20260812 03:05:51.284631 29818 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:34127
I20260812 03:05:51.284700 30086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:34127 every 8 connection(s)
I20260812 03:05:51.285511 29818 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:05:51.293743 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 29818
I20260812 03:05:51.293866 29891 log.cc:824] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c: Log is configured to *not* fsync() on all Append() calls
I20260812 03:05:51.294234 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:32923
--local_ip_for_outbound_sockets=127.5.96.130
--tserver_master_addrs=127.5.96.190:33641
--webserver_port=39461
--webserver_interface=127.5.96.130
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:05:51 dist-test-slave-6bbx krb5kdc[28808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503951, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:05:51.301752 30093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:51.287347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.129:50677)
0812 03:05:51.287835 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:51.287887 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:51.288650 (+ 763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:51.289960 (+ 1310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:51.289967 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:51.290313 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:05:51.290753 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:51.290766 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.292697 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.292700 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:51.293563 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:51.293575 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.294372 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.294377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:51.294563 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:51.295169 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:05:51.295185 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:05:51.296635 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:05:51.298806 (+ 2171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:51.298812 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:51.298825 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:51.299066 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:51.299403 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:51.299406 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:51.299407 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:51.299540 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:51.300120 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:05:51.300125 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:05:51.300394 (+ 269us) client_negotiation.cc:769] Sending connection context
0812 03:05:51.300553 (+ 159us) client_negotiation.cc:240] Negotiation successful
0812 03:05:51.300874 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":113,"threads_started":1}
I20260812 03:05:51.301740 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:51.287426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:50677 (local address 127.5.96.190:33641)
0812 03:05:51.287979 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:51.287998 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:51.288044 (+ 46us) server_negotiation.cc:407] Connection header received
0812 03:05:51.289111 (+ 1067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:51.289130 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:51.289537 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:51.289911 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:51.290926 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.292524 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.293691 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.294221 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.294403 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:51.296825 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:51.296831 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:51.296847 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:51.296880 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:51.298693 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:51.299183 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:51.299189 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:51.299193 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:51.299312 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:51.299636 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:51.299640 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:51.299641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:51.299998 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:51.300123 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:51.300701 (+ 578us) server_negotiation.cc:299] Negotiation successful
0812 03:05:51.300960 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":129,"threads_started":1}
I20260812 03:05:51.301972 29891 tablet_bootstrap.cc:492] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:05:51.302388 29891 tablet_bootstrap.cc:492] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c: Bootstrap complete.
I20260812 03:05:51.303691 29891 ts_tablet_manager.cc:1397] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c: Time spent bootstrapping tablet: real 0.028s user 0.012s sys 0.011s
I20260812 03:05:51.304184 30088 heartbeater.cc:347] Connected to a master server at 127.5.96.190:33641
I20260812 03:05:51.304402 30088 heartbeater.cc:464] Registering TS with master...
I20260812 03:05:51.304839 30088 heartbeater.cc:511] Master 127.5.96.190:33641 requested a full tablet report, sending...
I20260812 03:05:51.306555 29891 raft_consensus.cc:348] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } } peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } }
I20260812 03:05:51.307026 29744 ts_manager.cc:194] Registered new tserver with Master: 9d79d5dc5e114111b110c89a2b06456c (127.5.96.129:34127)
I20260812 03:05:51.307085 29891 raft_consensus.cc:729] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d79d5dc5e114111b110c89a2b06456c, State: Initialized, Role: FOLLOWER
I20260812 03:05:51.307582 29891 consensus_queue.cc:260] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } } peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } }
I20260812 03:05:51.308751 29891 ts_tablet_manager.cc:1428] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260812 03:05:51.310416 29744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.129@KRBTEST.COM'} at 127.5.96.129:50677
I20260812 03:05:51.312768 30088 heartbeater.cc:503] Master 127.5.96.190:33641 was elected leader, sending a full tablet report...
Aug 12 03:05:51 dist-test-slave-6bbx krb5kdc[28808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503951, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for kudu/127.5.96.129@KRBTEST.COM
I20260812 03:05:51.319995 30106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:51.309963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45032 (local address 127.5.96.129:34127)
0812 03:05:51.310287 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:51.310298 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:51.310758 (+ 460us) server_negotiation.cc:407] Connection header received
0812 03:05:51.310933 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:51.310939 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:51.311137 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:51.311273 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:51.311945 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.312938 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.314122 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.314493 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.314576 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:51.316907 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:51.316916 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:51.316925 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:51.316947 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:51.318295 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:51.318785 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:51.318788 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:51.318790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:51.318862 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:51.319099 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:51.319101 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:51.319102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:51.319299 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:51.319393 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:51.319752 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 03:05:51.319882 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":89,"threads_started":1}
I20260812 03:05:51.320020 30101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:51.309889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:34127 (local address 127.0.0.1:45032)
0812 03:05:51.310623 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:51.310666 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:51.310804 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:51.311268 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:51.311274 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:51.311689 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:05:51.311820 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:51.311828 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.313055 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.313059 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:51.314024 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:51.314034 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.314598 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.314601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:51.314711 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:51.315203 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:05:51.315218 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:05:51.316781 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:05:51.318394 (+ 1613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:51.318402 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:51.318408 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:51.318678 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:51.318934 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:51.318937 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:51.318939 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:51.319028 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:51.319393 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:05:51.319400 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:05:51.319515 (+ 115us) client_negotiation.cc:769] Sending connection context
0812 03:05:51.319647 (+ 132us) client_negotiation.cc:240] Negotiation successful
0812 03:05:51.319852 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":99,"threads_started":1}
I20260812 03:05:51.320972 29975 tablet_service.cc:1514] Processing DeleteTablet for tablet e44de8fccc79450a81c4e1b43116f761 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:05:50 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:45032
I20260812 03:05:51.321710 30107 tablet_replica.cc:330] stopping tablet replica
I20260812 03:05:51.321993 30107 raft_consensus.cc:2229] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:05:51.322247 30107 raft_consensus.cc:2258] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:05:51.323138 30107 ts_tablet_manager.cc:1905] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:05:51.326702 30107 ts_tablet_manager.cc:1918] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:05:51.326792 30107 log.cc:1196] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/wal/wals/e44de8fccc79450a81c4e1b43116f761
I20260812 03:05:51.327028 30107 ts_tablet_manager.cc:1939] T e44de8fccc79450a81c4e1b43116f761 P 9d79d5dc5e114111b110c89a2b06456c: Deleting consensus metadata
I20260812 03:05:51.328576 29732 catalog_manager.cc:4841] TS 9d79d5dc5e114111b110c89a2b06456c (127.5.96.129:34127): tablet e44de8fccc79450a81c4e1b43116f761 (table test-table [id=abbfc5196fed45809eba4e55ec8b0040]) successfully deleted
W20260812 03:05:51.410307 30098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:05:51.410526 30098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:05:51.410563 30098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:05:51.414398 30098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:05:51.414445 30098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:05:51.414497 30098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:05:51.419358 30098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:32923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=39461
--tserver_master_addrs=127.5.96.190:33641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:05:51.420246 30098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:05:51.421133 30098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:05:51.430387 30119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:51.430467 30098 server_base.cc:1034] running on GCE node
W20260812 03:05:51.430382 30116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:51.430400 30115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:51.431035 30098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:05:51.431597 30098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:05:51.432749 30098 hybrid_clock.cc:648] HybridClock initialized: now 1786503951432730 us; error 42 us; skew 500 ppm
Aug 12 03:05:51 dist-test-slave-6bbx krb5kdc[28808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503951, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:05:51.435271 30098 init.cc:377] Logged in from keytab as kudu/127.5.96.130@KRBTEST.COM (short username kudu)
I20260812 03:05:51.436123 30098 webserver.cc:460] Webserver started at http://127.5.96.130:39461/ using document root <none> and password file <none>
I20260812 03:05:51.436645 30098 fs_manager.cc:359] Metadata directory not provided
I20260812 03:05:51.436724 30098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:05:51.440343 30098 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:05:51.442270 30129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:51.443434 30098 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:05:51.443537 30098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0"
format_stamp: "Formatted at 2026-08-12 03:05:50 on dist-test-slave-6bbx"
server_key: "0735851329e2d18cfd3b76501241dd96"
server_key_iv: "762717fba17354a8fd9ba4c4191cc3f8"
server_key_version: "encryptionkey@0"
I20260812 03:05:51.444087 30098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:05:51.457114 30098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:05:51.459918 30098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:05:51.460076 30098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:05:51.460577 30098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:05:51.461321 30098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:05:51.461366 30098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:51.461406 30098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:05:51.461436 30098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:51.469478 30098 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:32923
I20260812 03:05:51.469524 30323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:32923 every 8 connection(s)
I20260812 03:05:51.470482 30098 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:05:51.472462 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 30098
I20260812 03:05:51.472812 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:45251
--local_ip_for_outbound_sockets=127.5.96.131
--tserver_master_addrs=127.5.96.190:33641
--webserver_port=36517
--webserver_interface=127.5.96.131
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:05:51 dist-test-slave-6bbx krb5kdc[28808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503951, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:05:51.484287 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:51.472145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:52257 (local address 127.5.96.190:33641)
0812 03:05:51.472304 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:51.472309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:51.473333 (+ 1024us) server_negotiation.cc:407] Connection header received
0812 03:05:51.474157 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:51.474161 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:51.474219 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:51.474307 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:51.475809 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.476332 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.477856 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.478108 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.478216 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:51.480735 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:51.480738 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:51.480740 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:51.480778 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:51.482031 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:51.482506 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:51.482509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:51.482510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:51.482590 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:51.482962 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:51.482967 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:51.483002 (+ 35us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:51.483279 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:51.483386 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:51.484043 (+ 657us) server_negotiation.cc:299] Negotiation successful
0812 03:05:51.484158 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 03:05:51.485296 30326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:51.472477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.130:52257)
0812 03:05:51.473176 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:51.473225 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:51.473966 (+ 741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:51.474483 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:51.474492 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:51.474946 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:05:51.475590 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:51.475610 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.476472 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.476476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:51.477727 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:51.477741 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.478286 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.478290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:51.478481 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:51.478998 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:05:51.479018 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:05:51.480570 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:05:51.482148 (+ 1578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:51.482154 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:51.482167 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:51.482412 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:51.482689 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:51.482692 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:51.482694 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:51.482863 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:51.483448 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:05:51.483455 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:05:51.483776 (+ 321us) client_negotiation.cc:769] Sending connection context
0812 03:05:51.483996 (+ 220us) client_negotiation.cc:240] Negotiation successful
0812 03:05:51.484328 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":123,"threads_started":1}
I20260812 03:05:51.486370 30324 heartbeater.cc:347] Connected to a master server at 127.5.96.190:33641
I20260812 03:05:51.486577 30324 heartbeater.cc:464] Registering TS with master...
I20260812 03:05:51.487012 30324 heartbeater.cc:511] Master 127.5.96.190:33641 requested a full tablet report, sending...
I20260812 03:05:51.487962 29744 ts_manager.cc:194] Registered new tserver with Master: 1a0f8e3c4c9b44ada0371e5a305f2bd0 (127.5.96.130:32923)
I20260812 03:05:51.488524 29744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.130@KRBTEST.COM'} at 127.5.96.130:52257
W20260812 03:05:51.585191 30331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:05:51.585422 30331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:05:51.585480 30331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:05:51.589347 30331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:05:51.589396 30331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:05:51.589468 30331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:05:51.594434 30331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:45251
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=36517
--tserver_master_addrs=127.5.96.190:33641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:05:51.595355 30331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:05:51.596316 30331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:05:51.605201 30338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:51.605206 30339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:51.605242 30331 server_base.cc:1034] running on GCE node
W20260812 03:05:51.605191 30342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:51.605755 30331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:05:51.606333 30331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:05:51.607486 30331 hybrid_clock.cc:648] HybridClock initialized: now 1786503951607472 us; error 27 us; skew 500 ppm
Aug 12 03:05:51 dist-test-slave-6bbx krb5kdc[28808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503951, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:05:51.610054 30331 init.cc:377] Logged in from keytab as kudu/127.5.96.131@KRBTEST.COM (short username kudu)
I20260812 03:05:51.610853 30331 webserver.cc:460] Webserver started at http://127.5.96.131:36517/ using document root <none> and password file <none>
I20260812 03:05:51.611346 30331 fs_manager.cc:359] Metadata directory not provided
I20260812 03:05:51.611428 30331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:05:51.614954 30331 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260812 03:05:51.616844 30353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:51.617928 30331 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:05:51.618031 30331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "96e4c609213140e993ce7f1a08de5ec0"
format_stamp: "Formatted at 2026-08-12 03:05:50 on dist-test-slave-6bbx"
server_key: "2fae027ca5002e691801d719b4490aec"
server_key_iv: "834497e55b4ce0820f7f17c6e9e4de70"
server_key_version: "encryptionkey@0"
I20260812 03:05:51.618505 30331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:05:51.632295 30331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:05:51.635058 30331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:05:51.635211 30331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:05:51.635756 30331 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:05:51.636529 30331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:05:51.636583 30331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:51.636634 30331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:05:51.636658 30331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:51.645232 30331 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:45251
I20260812 03:05:51.645256 30540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:45251 every 8 connection(s)
I20260812 03:05:51.646109 30331 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:05:51.650153 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 30331
Aug 12 03:05:51 dist-test-slave-6bbx krb5kdc[28808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503951, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:05:51.658383 30551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:51.650803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39240 (local address 127.5.96.190:33641)
0812 03:05:51.651019 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:51.651022 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:51.651179 (+ 157us) server_negotiation.cc:407] Connection header received
0812 03:05:51.651285 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:51.651288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:51.651332 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:51.651424 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:51.652166 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.652654 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.653634 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.653840 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.653915 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:51.655214 (+ 1299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:51.655218 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:51.655229 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:51.655256 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:51.656836 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:51.657235 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:51.657239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:51.657241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:51.657323 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:51.657579 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:51.657582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:51.657584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:51.657762 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:51.657953 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:51.658107 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 03:05:51.658220 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":71,"threads_started":1}
I20260812 03:05:51.659201 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:51.647759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:51451 (local address 127.5.96.190:33641)
0812 03:05:51.647903 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:51.647908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:51.648947 (+ 1039us) server_negotiation.cc:407] Connection header received
0812 03:05:51.649726 (+ 779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:51.649729 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:51.649775 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:51.649843 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:51.651192 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.651849 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.653153 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.653365 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.653439 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:51.656049 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:51.656052 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:51.656054 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:51.656087 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:51.657215 (+ 1128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:51.657669 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:51.657673 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:51.657675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:51.657756 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:51.658067 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:51.658070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:51.658071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:51.658232 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:51.658312 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:51.658942 (+ 630us) server_negotiation.cc:299] Negotiation successful
0812 03:05:51.659059 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:05:51.660022 30549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:51.648173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.131:51451)
0812 03:05:51.648820 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:51.648859 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:51.649553 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:51.650018 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:51.650028 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:51.650475 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:05:51.651047 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:51.651066 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.651968 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.651972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:51.653042 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:51.653053 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.653495 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.653499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:51.653696 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:51.654207 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:05:51.654228 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:05:51.655903 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:05:51.657320 (+ 1417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:51.657325 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:51.657338 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:51.657577 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:51.657838 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:51.657841 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:51.657843 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:51.657972 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:51.658431 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:05:51.658437 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:05:51.658709 (+ 272us) client_negotiation.cc:769] Sending connection context
0812 03:05:51.658897 (+ 188us) client_negotiation.cc:240] Negotiation successful
0812 03:05:51.659182 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":98,"threads_started":1}
I20260812 03:05:51.661056 30541 heartbeater.cc:347] Connected to a master server at 127.5.96.190:33641
I20260812 03:05:51.661302 30541 heartbeater.cc:464] Registering TS with master...
I20260812 03:05:51.661732 30541 heartbeater.cc:511] Master 127.5.96.190:33641 requested a full tablet report, sending...
I20260812 03:05:51.662642 29744 ts_manager.cc:194] Registered new tserver with Master: 96e4c609213140e993ce7f1a08de5ec0 (127.5.96.131:45251)
I20260812 03:05:51.663163 29744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.131@KRBTEST.COM'} at 127.5.96.131:51451
I20260812 03:05:51.671504 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:05:51.675354 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:51.672085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39252 (local address 127.5.96.190:33641)
0812 03:05:51.672197 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:51.672201 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:51.672357 (+ 156us) server_negotiation.cc:407] Connection header received
0812 03:05:51.672501 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:51.672504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:51.672591 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:51.672669 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:05:51.672881 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.673338 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.674139 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.674341 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.674422 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:51.674701 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:05:51.675025 (+ 324us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:05:51.675123 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:51.675194 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 03:05:51.675240 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:05:51.675932 29744 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:39252:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:05:51.679054 29744 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:05:51.686123 29975 tablet_service.cc:1467] Processing CreateTablet for tablet 69ff76ea2b5f43919494e9b47b255f50 (DEFAULT_TABLE table=test-table [id=4c9f622963d84cfab19888ad0c7585c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:05:51.686765 29975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69ff76ea2b5f43919494e9b47b255f50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:05:51.689592 29891 tablet_bootstrap.cc:492] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c: Bootstrap starting.
I20260812 03:05:51.689602 30559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:51.685232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:45251 (local address 127.0.0.1:45466)
0812 03:05:51.685526 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:51.685539 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:51.685624 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:51.686207 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:51.686210 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:51.686225 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:05:51.686295 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:51.686301 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.687345 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.687348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:51.688452 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:51.688461 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.689202 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.689205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:51.689297 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:51.689338 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 03:05:51.689410 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 03:05:51.689476 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":44,"thread_start_us":75,"threads_started":1}
I20260812 03:05:51.690097 30101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:51.684973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:32923 (local address 127.0.0.1:53304)
0812 03:05:51.685143 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:51.685154 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:51.685218 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:51.686102 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:51.686105 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:51.686124 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:05:51.686210 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:51.686216 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.687578 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.687582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:51.688687 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:51.688699 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.689774 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.689778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:51.689873 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:51.689894 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 03:05:51.689941 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 03:05:51.690012 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":18}
I20260812 03:05:51.690392 29891 tablet_bootstrap.cc:654] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c: Neither blocks nor log segments found. Creating new log.
I20260812 03:05:51.690624 30560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:51.685163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53304 (local address 127.5.96.130:32923)
0812 03:05:51.685630 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:51.685641 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:51.685663 (+ 22us) server_negotiation.cc:407] Connection header received
0812 03:05:51.685735 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:51.685740 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:51.685950 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:51.686105 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:05:51.686400 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.687433 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.688846 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.689635 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.690258 (+ 623us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:51.690386 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:51.690425 (+ 39us) server_negotiation.cc:299] Negotiation successful
0812 03:05:51.690496 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":157,"threads_started":1}
I20260812 03:05:51.691816 29891 tablet_bootstrap.cc:492] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c: No bootstrap required, opened a new log
I20260812 03:05:51.691910 29891 ts_tablet_manager.cc:1397] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:05:51.692329 29891 raft_consensus.cc:348] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } } peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } }
I20260812 03:05:51.692425 29891 raft_consensus.cc:374] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:05:51.692461 29891 raft_consensus.cc:729] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d79d5dc5e114111b110c89a2b06456c, State: Initialized, Role: FOLLOWER
I20260812 03:05:51.692584 29891 consensus_queue.cc:260] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } } peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } }
I20260812 03:05:51.692827 29891 ts_tablet_manager.cc:1428] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:05:51.693265 30219 tablet_service.cc:1467] Processing CreateTablet for tablet 69ff76ea2b5f43919494e9b47b255f50 (DEFAULT_TABLE table=test-table [id=4c9f622963d84cfab19888ad0c7585c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:05:51.694317 30219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69ff76ea2b5f43919494e9b47b255f50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:05:51.695900 30562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:51.685461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45466 (local address 127.5.96.131:45251)
0812 03:05:51.685848 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:51.685857 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:51.685874 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:05:51.685925 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:51.685928 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:51.686068 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:51.686408 (+ 340us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:05:51.686508 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.687227 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.688565 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.689091 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.689186 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:51.689317 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:51.689448 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 03:05:51.695792 (+ 6344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":147,"threads_started":1}
I20260812 03:05:51.698112 30436 tablet_service.cc:1467] Processing CreateTablet for tablet 69ff76ea2b5f43919494e9b47b255f50 (DEFAULT_TABLE table=test-table [id=4c9f622963d84cfab19888ad0c7585c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:05:51.698980 30436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69ff76ea2b5f43919494e9b47b255f50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:05:51.701114 30564 tablet_bootstrap.cc:492] T 69ff76ea2b5f43919494e9b47b255f50 P 1a0f8e3c4c9b44ada0371e5a305f2bd0: Bootstrap starting.
I20260812 03:05:51.703810 30564 tablet_bootstrap.cc:654] T 69ff76ea2b5f43919494e9b47b255f50 P 1a0f8e3c4c9b44ada0371e5a305f2bd0: Neither blocks nor log segments found. Creating new log.
I20260812 03:05:51.704658 30564 log.cc:824] T 69ff76ea2b5f43919494e9b47b255f50 P 1a0f8e3c4c9b44ada0371e5a305f2bd0: Log is configured to *not* fsync() on all Append() calls
I20260812 03:05:51.705818 30566 tablet_bootstrap.cc:492] T 69ff76ea2b5f43919494e9b47b255f50 P 96e4c609213140e993ce7f1a08de5ec0: Bootstrap starting.
I20260812 03:05:51.706566 30564 tablet_bootstrap.cc:492] T 69ff76ea2b5f43919494e9b47b255f50 P 1a0f8e3c4c9b44ada0371e5a305f2bd0: No bootstrap required, opened a new log
I20260812 03:05:51.706732 30564 ts_tablet_manager.cc:1397] T 69ff76ea2b5f43919494e9b47b255f50 P 1a0f8e3c4c9b44ada0371e5a305f2bd0: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260812 03:05:51.707494 30566 tablet_bootstrap.cc:654] T 69ff76ea2b5f43919494e9b47b255f50 P 96e4c609213140e993ce7f1a08de5ec0: Neither blocks nor log segments found. Creating new log.
I20260812 03:05:51.708453 30566 log.cc:824] T 69ff76ea2b5f43919494e9b47b255f50 P 96e4c609213140e993ce7f1a08de5ec0: Log is configured to *not* fsync() on all Append() calls
I20260812 03:05:51.709673 30564 raft_consensus.cc:348] T 69ff76ea2b5f43919494e9b47b255f50 P 1a0f8e3c4c9b44ada0371e5a305f2bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } } peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } }
I20260812 03:05:51.709887 30564 raft_consensus.cc:374] T 69ff76ea2b5f43919494e9b47b255f50 P 1a0f8e3c4c9b44ada0371e5a305f2bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:05:51.709942 30564 raft_consensus.cc:729] T 69ff76ea2b5f43919494e9b47b255f50 P 1a0f8e3c4c9b44ada0371e5a305f2bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a0f8e3c4c9b44ada0371e5a305f2bd0, State: Initialized, Role: FOLLOWER
I20260812 03:05:51.710194 30566 tablet_bootstrap.cc:492] T 69ff76ea2b5f43919494e9b47b255f50 P 96e4c609213140e993ce7f1a08de5ec0: No bootstrap required, opened a new log
I20260812 03:05:51.710358 30566 ts_tablet_manager.cc:1397] T 69ff76ea2b5f43919494e9b47b255f50 P 96e4c609213140e993ce7f1a08de5ec0: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 03:05:51.710342 30564 consensus_queue.cc:260] T 69ff76ea2b5f43919494e9b47b255f50 P 1a0f8e3c4c9b44ada0371e5a305f2bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } } peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } }
I20260812 03:05:51.710940 30324 heartbeater.cc:503] Master 127.5.96.190:33641 was elected leader, sending a full tablet report...
I20260812 03:05:51.711126 30564 ts_tablet_manager.cc:1428] T 69ff76ea2b5f43919494e9b47b255f50 P 1a0f8e3c4c9b44ada0371e5a305f2bd0: Time spent starting tablet: real 0.004s user 0.005s sys 0.001s
I20260812 03:05:51.713760 30566 raft_consensus.cc:348] T 69ff76ea2b5f43919494e9b47b255f50 P 96e4c609213140e993ce7f1a08de5ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } } peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } }
I20260812 03:05:51.714027 30566 raft_consensus.cc:374] T 69ff76ea2b5f43919494e9b47b255f50 P 96e4c609213140e993ce7f1a08de5ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:05:51.714100 30566 raft_consensus.cc:729] T 69ff76ea2b5f43919494e9b47b255f50 P 96e4c609213140e993ce7f1a08de5ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96e4c609213140e993ce7f1a08de5ec0, State: Initialized, Role: FOLLOWER
I20260812 03:05:51.714463 30566 consensus_queue.cc:260] T 69ff76ea2b5f43919494e9b47b255f50 P 96e4c609213140e993ce7f1a08de5ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } } peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } }
I20260812 03:05:51.714994 30541 heartbeater.cc:503] Master 127.5.96.190:33641 was elected leader, sending a full tablet report...
I20260812 03:05:51.715322 30566 ts_tablet_manager.cc:1428] T 69ff76ea2b5f43919494e9b47b255f50 P 96e4c609213140e993ce7f1a08de5ec0: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260812 03:05:51.721351 30325 tablet.cc:2369] T 69ff76ea2b5f43919494e9b47b255f50 P 1a0f8e3c4c9b44ada0371e5a305f2bd0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:05:51.786787 30092 tablet.cc:2369] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:05:51.838505 30100 raft_consensus.cc:482] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:05:51.838726 30100 raft_consensus.cc:504] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } } peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } }
I20260812 03:05:51.839895 30100 leader_election.cc:288] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a0f8e3c4c9b44ada0371e5a305f2bd0 (127.5.96.130:32923), 96e4c609213140e993ce7f1a08de5ec0 (127.5.96.131:45251)
I20260812 03:05:51.843528 30560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:51.840196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:33113 (local address 127.5.96.130:32923)
0812 03:05:51.840320 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:51.840324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:51.840436 (+ 112us) server_negotiation.cc:407] Connection header received
0812 03:05:51.840608 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:51.840611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:51.840655 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:51.840717 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:05:51.840973 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.841463 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.842537 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.842992 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.843090 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:51.843143 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:51.843349 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 03:05:51.843416 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:05:51.843550 30575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:51.840090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:32923 (local address 127.5.96.129:33113)
0812 03:05:51.840360 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:51.840372 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:51.840489 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:51.840736 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:51.840739 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:51.840757 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:05:51.840871 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:51.840877 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.841577 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.841580 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:51.842417 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:51.842427 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.843101 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.843105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:51.843206 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:51.843246 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 03:05:51.843320 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 03:05:51.843396 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":95,"threads_started":1}
I20260812 03:05:51.843607 30562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:51.840200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:57091 (local address 127.5.96.131:45251)
0812 03:05:51.840325 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:51.840330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:51.840648 (+ 318us) server_negotiation.cc:407] Connection header received
0812 03:05:51.840772 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:51.840774 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:51.840817 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:51.840877 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:05:51.841093 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.841651 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.842699 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.843101 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.843207 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:51.843255 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:51.843428 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 03:05:51.843491 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:05:51.843847 30579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:51.840090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:45251 (local address 127.5.96.129:57091)
0812 03:05:51.840564 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:51.840576 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:51.840671 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:51.840896 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:51.840899 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:51.840915 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:05:51.841002 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:51.841007 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.841740 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.841743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:51.842546 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:51.842557 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.843215 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.843219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:51.843321 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:51.843341 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 03:05:51.843405 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 03:05:51.843471 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":253,"thread_start_us":71,"threads_started":1}
I20260812 03:05:51.844326 30256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69ff76ea2b5f43919494e9b47b255f50" candidate_uuid: "9d79d5dc5e114111b110c89a2b06456c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" is_pre_election: true
I20260812 03:05:51.844456 30465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69ff76ea2b5f43919494e9b47b255f50" candidate_uuid: "9d79d5dc5e114111b110c89a2b06456c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96e4c609213140e993ce7f1a08de5ec0" is_pre_election: true
I20260812 03:05:51.844614 30256 raft_consensus.cc:2454] T 69ff76ea2b5f43919494e9b47b255f50 P 1a0f8e3c4c9b44ada0371e5a305f2bd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d79d5dc5e114111b110c89a2b06456c in term 0.
I20260812 03:05:51.844722 30465 raft_consensus.cc:2454] T 69ff76ea2b5f43919494e9b47b255f50 P 96e4c609213140e993ce7f1a08de5ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d79d5dc5e114111b110c89a2b06456c in term 0.
I20260812 03:05:51.845060 29880 leader_election.cc:302] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a0f8e3c4c9b44ada0371e5a305f2bd0, 9d79d5dc5e114111b110c89a2b06456c; no voters:
I20260812 03:05:51.845319 30100 raft_consensus.cc:2781] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:05:51.845397 30100 raft_consensus.cc:482] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:05:51.845428 30100 raft_consensus.cc:3037] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:05:51.846398 30100 raft_consensus.cc:504] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } } peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } }
I20260812 03:05:51.846701 30100 leader_election.cc:288] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 1 election: Requested vote from peers 1a0f8e3c4c9b44ada0371e5a305f2bd0 (127.5.96.130:32923), 96e4c609213140e993ce7f1a08de5ec0 (127.5.96.131:45251)
I20260812 03:05:51.847009 30256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69ff76ea2b5f43919494e9b47b255f50" candidate_uuid: "9d79d5dc5e114111b110c89a2b06456c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0"
I20260812 03:05:51.847050 30465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69ff76ea2b5f43919494e9b47b255f50" candidate_uuid: "9d79d5dc5e114111b110c89a2b06456c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96e4c609213140e993ce7f1a08de5ec0"
I20260812 03:05:51.847115 30256 raft_consensus.cc:3037] T 69ff76ea2b5f43919494e9b47b255f50 P 1a0f8e3c4c9b44ada0371e5a305f2bd0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:05:51.847150 30465 raft_consensus.cc:3037] T 69ff76ea2b5f43919494e9b47b255f50 P 96e4c609213140e993ce7f1a08de5ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:05:51.848160 30465 raft_consensus.cc:2454] T 69ff76ea2b5f43919494e9b47b255f50 P 96e4c609213140e993ce7f1a08de5ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d79d5dc5e114111b110c89a2b06456c in term 1.
I20260812 03:05:51.848268 30256 raft_consensus.cc:2454] T 69ff76ea2b5f43919494e9b47b255f50 P 1a0f8e3c4c9b44ada0371e5a305f2bd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d79d5dc5e114111b110c89a2b06456c in term 1.
I20260812 03:05:51.848416 29887 leader_election.cc:302] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96e4c609213140e993ce7f1a08de5ec0, 9d79d5dc5e114111b110c89a2b06456c; no voters:
I20260812 03:05:51.848599 30100 raft_consensus.cc:2781] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:05:51.848760 30100 raft_consensus.cc:686] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 1 LEADER]: Becoming Leader. State: Replica: 9d79d5dc5e114111b110c89a2b06456c, State: Running, Role: LEADER
I20260812 03:05:51.848999 30100 consensus_queue.cc:237] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } } peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } }
I20260812 03:05:51.851691 29744 catalog_manager.cc:5495] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d79d5dc5e114111b110c89a2b06456c (127.5.96.129). New cstate: current_term: 1 leader_uuid: "9d79d5dc5e114111b110c89a2b06456c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } health_report { overall_health: HEALTHY } } }
I20260812 03:05:51.877074 30582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:51.873918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45040 (local address 127.5.96.129:34127)
0812 03:05:51.874170 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:51.874177 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:51.874188 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:05:51.874261 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:51.874263 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:51.874314 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:51.874409 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:05:51.874574 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.875056 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.875826 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:51.876043 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:51.876119 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:51.876308 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:05:51.876777 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:05:51.876857 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:51.876911 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 03:05:51.876955 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":107,"threads_started":1}
I20260812 03:05:51.880600 30256 raft_consensus.cc:1261] T 69ff76ea2b5f43919494e9b47b255f50 P 1a0f8e3c4c9b44ada0371e5a305f2bd0 [term 1 FOLLOWER]: Refusing update from remote peer 9d79d5dc5e114111b110c89a2b06456c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:05:51.880601 30465 raft_consensus.cc:1261] T 69ff76ea2b5f43919494e9b47b255f50 P 96e4c609213140e993ce7f1a08de5ec0 [term 1 FOLLOWER]: Refusing update from remote peer 9d79d5dc5e114111b110c89a2b06456c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:05:51.881273 30100 consensus_queue.cc:1035] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [LEADER]: Connected to new peer: Peer: permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:05:51.881445 30580 consensus_queue.cc:1035] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:05:51.888031 30584 mvcc.cc:205] Tried to move back new op lower bound from 7317520186897285120 to 7317520186775945216. Current Snapshot: MvccSnapshot[applied={T|T < 7317520186897285120}]
I20260812 03:05:51.889344 30585 mvcc.cc:205] Tried to move back new op lower bound from 7317520186897285120 to 7317520186775945216. Current Snapshot: MvccSnapshot[applied={T|T < 7317520186897285120}]
I20260812 03:05:51.890213 30586 mvcc.cc:205] Tried to move back new op lower bound from 7317520186897285120 to 7317520186775945216. Current Snapshot: MvccSnapshot[applied={T|T < 7317520186897285120}]
I20260812 03:05:51.896008 29745 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:39252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:05:51.896142 29745 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:39252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:05:51.898192 29745 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16: Sending DeleteTablet for 3 replicas of tablet 69ff76ea2b5f43919494e9b47b255f50
I20260812 03:05:51.898877 29975 tablet_service.cc:1514] Processing DeleteTablet for tablet 69ff76ea2b5f43919494e9b47b255f50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:05:51 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:45032
I20260812 03:05:51.898834 30219 tablet_service.cc:1514] Processing DeleteTablet for tablet 69ff76ea2b5f43919494e9b47b255f50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:05:51 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:53304
I20260812 03:05:51.899195 30603 tablet_replica.cc:330] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c: stopping tablet replica
I20260812 03:05:51.899271 30604 tablet_replica.cc:330] T 69ff76ea2b5f43919494e9b47b255f50 P 1a0f8e3c4c9b44ada0371e5a305f2bd0: stopping tablet replica
I20260812 03:05:51.899274 30436 tablet_service.cc:1514] Processing DeleteTablet for tablet 69ff76ea2b5f43919494e9b47b255f50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:05:51 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:45466
I20260812 03:05:51.899467 30603 raft_consensus.cc:2229] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:05:51.899557 30604 raft_consensus.cc:2229] T 69ff76ea2b5f43919494e9b47b255f50 P 1a0f8e3c4c9b44ada0371e5a305f2bd0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:05:51.899868 30604 raft_consensus.cc:2258] T 69ff76ea2b5f43919494e9b47b255f50 P 1a0f8e3c4c9b44ada0371e5a305f2bd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:05:51.899858 30603 raft_consensus.cc:2258] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:05:51.899935 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 29818
I20260812 03:05:51.910140 30605 tablet_replica.cc:330] T 69ff76ea2b5f43919494e9b47b255f50 P 96e4c609213140e993ce7f1a08de5ec0: stopping tablet replica
I20260812 03:05:51.910420 30605 raft_consensus.cc:2229] T 69ff76ea2b5f43919494e9b47b255f50 P 96e4c609213140e993ce7f1a08de5ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:05:51.910665 30605 raft_consensus.cc:2258] T 69ff76ea2b5f43919494e9b47b255f50 P 96e4c609213140e993ce7f1a08de5ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:05:51.911007 30604 ts_tablet_manager.cc:1905] T 69ff76ea2b5f43919494e9b47b255f50 P 1a0f8e3c4c9b44ada0371e5a305f2bd0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:05:51.912796 30605 ts_tablet_manager.cc:1905] T 69ff76ea2b5f43919494e9b47b255f50 P 96e4c609213140e993ce7f1a08de5ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:05:51.914108 30604 ts_tablet_manager.cc:1918] T 69ff76ea2b5f43919494e9b47b255f50 P 1a0f8e3c4c9b44ada0371e5a305f2bd0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:05:51.914229 30604 log.cc:1196] T 69ff76ea2b5f43919494e9b47b255f50 P 1a0f8e3c4c9b44ada0371e5a305f2bd0: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/wal/wals/69ff76ea2b5f43919494e9b47b255f50
I20260812 03:05:51.914529 30604 ts_tablet_manager.cc:1939] T 69ff76ea2b5f43919494e9b47b255f50 P 1a0f8e3c4c9b44ada0371e5a305f2bd0: Deleting consensus metadata
I20260812 03:05:51.914857 30605 ts_tablet_manager.cc:1918] T 69ff76ea2b5f43919494e9b47b255f50 P 96e4c609213140e993ce7f1a08de5ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:05:51.914954 30605 log.cc:1196] T 69ff76ea2b5f43919494e9b47b255f50 P 96e4c609213140e993ce7f1a08de5ec0: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/wal/wals/69ff76ea2b5f43919494e9b47b255f50
I20260812 03:05:51.915236 30605 ts_tablet_manager.cc:1939] T 69ff76ea2b5f43919494e9b47b255f50 P 96e4c609213140e993ce7f1a08de5ec0: Deleting consensus metadata
I20260812 03:05:51.915692 29727 catalog_manager.cc:4841] TS 1a0f8e3c4c9b44ada0371e5a305f2bd0 (127.5.96.130:32923): tablet 69ff76ea2b5f43919494e9b47b255f50 (table test-table [id=4c9f622963d84cfab19888ad0c7585c1]) successfully deleted
I20260812 03:05:51.916282 29732 catalog_manager.cc:4841] TS 96e4c609213140e993ce7f1a08de5ec0 (127.5.96.131:45251): tablet 69ff76ea2b5f43919494e9b47b255f50 (table test-table [id=4c9f622963d84cfab19888ad0c7585c1]) successfully deleted
W20260812 03:05:51.920383 29732 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.96.129:34127 (error 108)
I20260812 03:05:51.920437 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 30098
I20260812 03:05:51.924121 30101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:51.923838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:34127 (local address 127.0.0.1:45042)
0812 03:05:51.923944 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:51.924026 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.129:34127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 03:05:51.925175 29732 catalog_manager.cc:4584] TS 9d79d5dc5e114111b110c89a2b06456c (127.5.96.129:34127): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 69ff76ea2b5f43919494e9b47b255f50: Network error: Client connection negotiation failed: client connection to 127.5.96.129:34127: connect: Connection refused (error 111)
I20260812 03:05:51.930876 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 30331
I20260812 03:05:51.942775 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 29703
I20260812 03:05:51.956503 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:33641
--webserver_interface=127.5.96.190
--webserver_port=33439
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:33641
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:05:52.064249 30609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:05:52.064471 30609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:05:52.064512 30609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:05:52.068401 30609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:05:52.068449 30609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:05:52.068478 30609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:05:52.068507 30609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:05:52.068532 30609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:05:52.073550 30609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:33641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:33641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=33439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:05:52.074461 30609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:05:52.075372 30609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:05:52.083244 30622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:52.083308 30620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:52.083308 30618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:52.083442 30609 server_base.cc:1034] running on GCE node
I20260812 03:05:52.084103 30609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:05:52.085059 30609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:05:52.086212 30609 hybrid_clock.cc:648] HybridClock initialized: now 1786503952086194 us; error 28 us; skew 500 ppm
Aug 12 03:05:52 dist-test-slave-6bbx krb5kdc[28808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503952, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:05:52.088742 30609 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:05:52.089531 30609 webserver.cc:460] Webserver started at http://127.5.96.190:33439/ using document root <none> and password file <none>
I20260812 03:05:52.090020 30609 fs_manager.cc:359] Metadata directory not provided
I20260812 03:05:52.090082 30609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:05:52.093593 30609 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:05:52.095465 30635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:52.096588 30609 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:05:52.096701 30609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "2fa15f69ad6b4d5a897c25d060030d16"
format_stamp: "Formatted at 2026-08-12 03:05:49 on dist-test-slave-6bbx"
server_key: "d2ae9c72ee319e852f913761276a6d56"
server_key_iv: "e5da01a115d3b1f1d7b7adfecde92416"
server_key_version: "encryptionkey@0"
I20260812 03:05:52.097172 30609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:05:52.110492 30609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:05:52.112895 30609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:05:52.113054 30609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:05:52.119681 30609 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:33641
I20260812 03:05:52.119946 30728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:33641 every 8 connection(s)
I20260812 03:05:52.120651 30609 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:05:52.122154 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 30609
I20260812 03:05:52.122545 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:34127
--local_ip_for_outbound_sockets=127.5.96.129
--tserver_master_addrs=127.5.96.190:33641
--webserver_port=42949
--webserver_interface=127.5.96.129
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:05:52.124780 30729 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:05:52.126297 30729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16: Bootstrap starting.
I20260812 03:05:52.143154 30729 log.cc:824] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16: Log is configured to *not* fsync() on all Append() calls
I20260812 03:05:52.153785 30729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:05:52.154100 30729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16: Bootstrap complete.
I20260812 03:05:52.157835 30729 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa15f69ad6b4d5a897c25d060030d16" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33641 } }
I20260812 03:05:52.158200 30729 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fa15f69ad6b4d5a897c25d060030d16, State: Initialized, Role: FOLLOWER
I20260812 03:05:52.158742 30729 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa15f69ad6b4d5a897c25d060030d16" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33641 } }
I20260812 03:05:52.158905 30729 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:05:52.158977 30729 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:05:52.159072 30729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [term 2 FOLLOWER]: Advancing to term 3
I20260812 03:05:52.160843 30729 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa15f69ad6b4d5a897c25d060030d16" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33641 } }
I20260812 03:05:52.161142 30729 leader_election.cc:302] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fa15f69ad6b4d5a897c25d060030d16; no voters:
I20260812 03:05:52.161427 30729 leader_election.cc:288] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 03:05:52.161512 30735 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 03:05:52.161720 30735 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [term 3 LEADER]: Becoming Leader. State: Replica: 2fa15f69ad6b4d5a897c25d060030d16, State: Running, Role: LEADER
I20260812 03:05:52.162007 30735 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa15f69ad6b4d5a897c25d060030d16" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33641 } }
I20260812 03:05:52.162621 30729 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:05:52.162930 30735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fa15f69ad6b4d5a897c25d060030d16. Latest consensus state: current_term: 3 leader_uuid: "2fa15f69ad6b4d5a897c25d060030d16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa15f69ad6b4d5a897c25d060030d16" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33641 } } }
I20260812 03:05:52.163120 30735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [sys.catalog]: This master's current role is: LEADER
I20260812 03:05:52.164232 30741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:05:52.164304 30736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "2fa15f69ad6b4d5a897c25d060030d16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa15f69ad6b4d5a897c25d060030d16" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33641 } } }
I20260812 03:05:52.164403 30736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fa15f69ad6b4d5a897c25d060030d16 [sys.catalog]: This master's current role is: LEADER
I20260812 03:05:52.169157 30741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:05:52.170099 30741 catalog_manager.cc:1259] Loaded cluster ID: bd50ef7171614e4bb1f091007661d8b5
I20260812 03:05:52.170159 30741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 03:05:52.171188 30741 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f3e82da69ad google::LogMessage::Fail() at ??:0
@ 0x7f3e82da988a google::LogMessage::SendToLog() at ??:0
@ 0x7f3e82da64de google::LogMessage::Flush() at ??:0
@ 0x7f3e82da7a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f3e8d6e5fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f3e8d6e6d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f3e8d6e3579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f3e8d727640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f3e84ff83d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f3e84ff8c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f3e84ffa3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f3e84fe9512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3e8662d6db start_thread at ??:0
@ 0x7f3e818a461f clone at ??:0
W20260812 03:05:52.245599 30732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:05:52.245807 30732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:05:52.245851 30732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:05:52.249784 30732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:05:52.249832 30732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:05:52.249902 30732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:05:52.254913 30732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:34127
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=42949
--tserver_master_addrs=127.5.96.190:33641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:05:52.255851 30732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:05:52.256776 30732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:05:52.265766 30801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:52.265944 30803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:52.266021 30732 server_base.cc:1034] running on GCE node
W20260812 03:05:52.265784 30798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:52.266513 30732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:05:52.267091 30732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:05:52.268250 30732 hybrid_clock.cc:648] HybridClock initialized: now 1786503952268221 us; error 47 us; skew 500 ppm
Aug 12 03:05:52 dist-test-slave-6bbx krb5kdc[28808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503952, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:05:52.270929 30732 init.cc:377] Logged in from keytab as kudu/127.5.96.129@KRBTEST.COM (short username kudu)
I20260812 03:05:52.271849 30732 webserver.cc:460] Webserver started at http://127.5.96.129:42949/ using document root <none> and password file <none>
I20260812 03:05:52.272372 30732 fs_manager.cc:359] Metadata directory not provided
I20260812 03:05:52.272432 30732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:05:52.276324 30732 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:05:52.278553 30813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:52.279887 30732 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:05:52.279999 30732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "9d79d5dc5e114111b110c89a2b06456c"
format_stamp: "Formatted at 2026-08-12 03:05:50 on dist-test-slave-6bbx"
server_key: "aeb91d1568259e24245d0b001a286900"
server_key_iv: "092bf30b3d04d7a83cc35ba1d007af63"
server_key_version: "encryptionkey@0"
I20260812 03:05:52.280503 30732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:05:52.306192 30732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:05:52.309134 30732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:05:52.309319 30732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:05:52.309883 30732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:05:52.310948 30826 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 03:05:52.316088 30732 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 03:05:52.316152 30732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260812 03:05:52.316206 30732 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 03:05:52.317823 30732 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 03:05:52.317869 30732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260812 03:05:52.318197 30826 tablet_bootstrap.cc:492] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c: Bootstrap starting.
I20260812 03:05:52.327570 30732 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:34127
I20260812 03:05:52.327749 31005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:34127 every 8 connection(s)
I20260812 03:05:52.328500 30732 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:05:52.329636 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 30732
I20260812 03:05:52.330056 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:32923
--local_ip_for_outbound_sockets=127.5.96.130
--tserver_master_addrs=127.5.96.190:33641
--webserver_port=39461
--webserver_interface=127.5.96.130
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:05:52.337828 30826 log.cc:824] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c: Log is configured to *not* fsync() on all Append() calls
I20260812 03:05:52.344338 30826 tablet_bootstrap.cc:492] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:05:52.344645 30826 tablet_bootstrap.cc:492] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c: Bootstrap complete.
I20260812 03:05:52.345549 30826 ts_tablet_manager.cc:1397] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c: Time spent bootstrapping tablet: real 0.028s user 0.021s sys 0.005s
I20260812 03:05:52.347832 30826 raft_consensus.cc:348] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } } peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } }
I20260812 03:05:52.348224 30826 raft_consensus.cc:729] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d79d5dc5e114111b110c89a2b06456c, State: Initialized, Role: FOLLOWER
I20260812 03:05:52.348740 30826 consensus_queue.cc:260] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } } peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } }
I20260812 03:05:52.349586 30826 ts_tablet_manager.cc:1428] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
W20260812 03:05:52.443607 31009 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:05:52.330508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.129:57813)
0812 03:05:52.331068 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:52.331157 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:52.332195 (+ 1038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:52.442559 (+110364us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":131,"threads_started":1}
W20260812 03:05:52.444253 31007 heartbeater.cc:644] Failed to heartbeat to 127.5.96.190:33641 (0 consecutive failures): Network error: Failed to ping master at 127.5.96.190:33641: Client connection negotiation failed: client connection to 127.5.96.190:33641: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 03:05:52.445032 31009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:52.444732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.129:44143)
0812 03:05:52.444882 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:52.444962 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 03:05:52.445750 31009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:52.445521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.129:38987)
0812 03:05:52.445633 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:52.445687 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 03:05:52.445930 31007 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:05:52.459574 31013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:05:52.459833 31013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:05:52.459883 31013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:05:52.463827 31013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:05:52.463873 31013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:05:52.463927 31013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:05:52.468839 31013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:32923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=39461
--tserver_master_addrs=127.5.96.190:33641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:05:52.469704 31013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:05:52.470654 31013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:05:52.479598 31027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:52.479617 31028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:52.479743 31013 server_base.cc:1034] running on GCE node
W20260812 03:05:52.479637 31032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:52.480211 31013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:05:52.480799 31013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:05:52.481951 31013 hybrid_clock.cc:648] HybridClock initialized: now 1786503952481942 us; error 27 us; skew 500 ppm
Aug 12 03:05:52 dist-test-slave-6bbx krb5kdc[28808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503952, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:05:52.484515 31013 init.cc:377] Logged in from keytab as kudu/127.5.96.130@KRBTEST.COM (short username kudu)
I20260812 03:05:52.485342 31013 webserver.cc:460] Webserver started at http://127.5.96.130:39461/ using document root <none> and password file <none>
I20260812 03:05:52.485850 31013 fs_manager.cc:359] Metadata directory not provided
I20260812 03:05:52.485916 31013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:05:52.489512 31013 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260812 03:05:52.491364 31040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:52.492517 31013 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 03:05:52.492624 31013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0"
format_stamp: "Formatted at 2026-08-12 03:05:50 on dist-test-slave-6bbx"
server_key: "0735851329e2d18cfd3b76501241dd96"
server_key_iv: "762717fba17354a8fd9ba4c4191cc3f8"
server_key_version: "encryptionkey@0"
I20260812 03:05:52.493116 31013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:05:52.510113 31013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:05:52.512770 31013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:05:52.512935 31013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:05:52.513494 31013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:05:52.514297 31013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:05:52.514346 31013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:52.514396 31013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:05:52.514425 31013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:52.522819 31013 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:32923
I20260812 03:05:52.522881 31223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:32923 every 8 connection(s)
I20260812 03:05:52.523756 31013 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:05:52.527027 31228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:52.525654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.130:49323)
0812 03:05:52.526279 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:52.526380 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":114,"threads_started":1}
W20260812 03:05:52.527426 31225 heartbeater.cc:644] Failed to heartbeat to 127.5.96.190:33641 (0 consecutive failures): Network error: Failed to ping master at 127.5.96.190:33641: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
I20260812 03:05:52.527987 31228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:52.527762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.130:49777)
0812 03:05:52.527858 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:52.527925 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 03:05:52.528147 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 31013
I20260812 03:05:52.528575 31228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:52.528365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.130:36269)
0812 03:05:52.528465 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:52.528516 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 03:05:52.528548 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:45251
--local_ip_for_outbound_sockets=127.5.96.131
--tserver_master_addrs=127.5.96.190:33641
--webserver_port=36517
--webserver_interface=127.5.96.131
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:38439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:05:52.528745 31225 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:05:52.637866 31232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:05:52.638091 31232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:05:52.638147 31232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:05:52.641983 31232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:05:52.642031 31232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:05:52.642122 31232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:05:52.647055 31232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:38439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:45251
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=36517
--tserver_master_addrs=127.5.96.190:33641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:05:52.647969 31232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:05:52.648880 31232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:05:52.657864 31239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:52.657917 31232 server_base.cc:1034] running on GCE node
W20260812 03:05:52.657881 31248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:05:52.658075 31242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:05:52.658557 31232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:05:52.659085 31232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:05:52.660240 31232 hybrid_clock.cc:648] HybridClock initialized: now 1786503952660225 us; error 26 us; skew 500 ppm
Aug 12 03:05:52 dist-test-slave-6bbx krb5kdc[28808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503952, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:05:52.662786 31232 init.cc:377] Logged in from keytab as kudu/127.5.96.131@KRBTEST.COM (short username kudu)
I20260812 03:05:52.663583 31232 webserver.cc:460] Webserver started at http://127.5.96.131:36517/ using document root <none> and password file <none>
I20260812 03:05:52.664093 31232 fs_manager.cc:359] Metadata directory not provided
I20260812 03:05:52.664155 31232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:05:52.667647 31232 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:05:52.669588 31258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:52.670746 31232 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 03:05:52.670856 31232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "96e4c609213140e993ce7f1a08de5ec0"
format_stamp: "Formatted at 2026-08-12 03:05:50 on dist-test-slave-6bbx"
server_key: "2fae027ca5002e691801d719b4490aec"
server_key_iv: "834497e55b4ce0820f7f17c6e9e4de70"
server_key_version: "encryptionkey@0"
I20260812 03:05:52.671356 31232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:05:52.698135 31232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:05:52.700820 31232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:05:52.700975 31232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:05:52.701512 31232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:05:52.702292 31232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:05:52.702338 31232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:52.702381 31232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:05:52.702409 31232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:05:52.710654 31232 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:45251
I20260812 03:05:52.710677 31452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:45251 every 8 connection(s)
I20260812 03:05:52.711580 31232 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:05:52.715016 31459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:52.713565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.131:57513)
0812 03:05:52.714191 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:52.714295 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":119,"threads_started":1}
I20260812 03:05:52.715189 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 31232
W20260812 03:05:52.715451 31455 heartbeater.cc:644] Failed to heartbeat to 127.5.96.190:33641 (0 consecutive failures): Network error: Failed to ping master at 127.5.96.190:33641: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
I20260812 03:05:52.716039 31459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:52.715811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.131:41005)
0812 03:05:52.715925 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:52.715979 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 03:05:52.716573 31459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:52.716416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.131:59135)
0812 03:05:52.716488 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:52.716526 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260812 03:05:52.716729 31455 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 03:05:53.446908 31462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:53.446484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.129:60873)
0812 03:05:53.446734 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:53.446787 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":75,"threads_started":1}
I20260812 03:05:53.529687 31463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:53.529269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.130:46939)
0812 03:05:53.529535 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:53.529586 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":85,"threads_started":1}
I20260812 03:05:53.717538 31465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:53.717180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.131:56181)
0812 03:05:53.717392 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:53.717439 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":73,"threads_started":1}
I20260812 03:05:54.031522 31466 raft_consensus.cc:482] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:05:54.031741 31466 raft_consensus.cc:504] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } } peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } }
I20260812 03:05:54.032843 31466 leader_election.cc:288] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1a0f8e3c4c9b44ada0371e5a305f2bd0 (127.5.96.130:32923), 96e4c609213140e993ce7f1a08de5ec0 (127.5.96.131:45251)
Aug 12 03:05:54 dist-test-slave-6bbx krb5kdc[28808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503952, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for kudu/127.5.96.131@KRBTEST.COM
Aug 12 03:05:54 dist-test-slave-6bbx krb5kdc[28808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503952, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for kudu/127.5.96.130@KRBTEST.COM
I20260812 03:05:54.046593 31468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:54.033019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:32923 (local address 127.5.96.129:40931)
0812 03:05:54.033323 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:54.033337 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:54.033446 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:54.034994 (+ 1548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:54.034999 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:54.035567 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:05:54.036130 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:54.036145 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:54.037366 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:54.037368 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:54.038443 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:54.038453 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:54.039113 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:54.039116 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:54.039275 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:54.039858 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:05:54.039880 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:05:54.041826 (+ 1946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:05:54.043805 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:54.043808 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:54.043809 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:54.044252 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:54.044605 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:54.044608 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:54.044610 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:54.044810 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:54.045496 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:05:54.045502 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:05:54.045840 (+ 338us) client_negotiation.cc:769] Sending connection context
0812 03:05:54.046059 (+ 219us) client_negotiation.cc:240] Negotiation successful
0812 03:05:54.046444 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":305,"spinlock_wait_cycles":18048,"thread_start_us":78,"threads_started":1}
I20260812 03:05:54.046631 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:54.033177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:40931 (local address 127.5.96.130:32923)
0812 03:05:54.033593 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:54.033609 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:54.033649 (+ 40us) server_negotiation.cc:407] Connection header received
0812 03:05:54.034316 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:54.034321 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:54.034645 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:54.034921 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:54.036286 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:54.037231 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:54.038556 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:54.039002 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:54.039154 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:54.042001 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:54.042007 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:54.042019 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:54.042050 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:54.043717 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:54.044366 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:54.044372 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:54.044377 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:54.044518 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:54.044901 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:54.044904 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:54.044906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:54.045381 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:54.045508 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:54.046177 (+ 669us) server_negotiation.cc:299] Negotiation successful
0812 03:05:54.046451 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":108,"threads_started":1}
I20260812 03:05:54.046631 31474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:54.033309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:52625 (local address 127.5.96.131:45251)
0812 03:05:54.033678 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0812 03:05:54.033699 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 03:05:54.033731 (+ 32us) server_negotiation.cc:407] Connection header received
0812 03:05:54.034356 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:05:54.034361 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:05:54.034644 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:05:54.034922 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:05:54.036303 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:54.037230 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:54.038545 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:05:54.039004 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:05:54.039155 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:54.041759 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:05:54.041764 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:05:54.041774 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:05:54.041803 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:05:54.043651 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:54.044366 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:54.044372 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:54.044377 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:54.044518 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:05:54.044901 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:05:54.044904 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:05:54.044906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:05:54.045381 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:05:54.045508 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 03:05:54.046177 (+ 669us) server_negotiation.cc:299] Negotiation successful
0812 03:05:54.046451 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":77,"threads_started":1}
I20260812 03:05:54.046592 31469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:54.033052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:45251 (local address 127.5.96.129:52625)
0812 03:05:54.033424 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:54.033438 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:05:54.033542 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:05:54.034993 (+ 1451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:05:54.034999 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:05:54.035566 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:05:54.036131 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:54.036145 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:54.037366 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:54.037369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:54.038437 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:05:54.038447 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:05:54.039381 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:05:54.039385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:05:54.039493 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:05:54.039967 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:05:54.039981 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:05:54.041570 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:05:54.043786 (+ 2216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:54.043793 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:54.043807 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:54.044252 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:54.044605 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:05:54.044608 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:05:54.044610 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:05:54.044810 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:05:54.045496 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:05:54.045502 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:05:54.045840 (+ 338us) client_negotiation.cc:769] Sending connection context
0812 03:05:54.046059 (+ 219us) client_negotiation.cc:240] Negotiation successful
0812 03:05:54.046376 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":197,"spinlock_wait_cycles":49280,"thread_start_us":74,"threads_started":1}
W20260812 03:05:54.049073 30820 leader_election.cc:341] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 96e4c609213140e993ce7f1a08de5ec0 (127.5.96.131:45251): Not found: Tablet not found: 69ff76ea2b5f43919494e9b47b255f50
W20260812 03:05:54.049252 30814 leader_election.cc:341] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1a0f8e3c4c9b44ada0371e5a305f2bd0 (127.5.96.130:32923): Not found: Tablet not found: 69ff76ea2b5f43919494e9b47b255f50
I20260812 03:05:54.049333 30814 leader_election.cc:302] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d79d5dc5e114111b110c89a2b06456c; no voters: 1a0f8e3c4c9b44ada0371e5a305f2bd0, 96e4c609213140e993ce7f1a08de5ec0
I20260812 03:05:54.049571 31466 raft_consensus.cc:2726] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:05:54.447892 31469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:54.447588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.129:56057)
0812 03:05:54.447743 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:54.447799 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 03:05:54.530711 31476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:54.530336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.130:53625)
0812 03:05:54.530558 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:54.530608 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":68,"threads_started":1}
I20260812 03:05:54.718472 31477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:54.718113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.131:35081)
0812 03:05:54.718341 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:54.718388 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":71,"threads_started":1}
I20260812 03:05:55.448956 31479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:55.448558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.129:34837)
0812 03:05:55.448805 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:55.448856 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":77,"threads_started":1}
I20260812 03:05:55.531658 31482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:55.531300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.130:55503)
0812 03:05:55.531522 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:55.531571 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":69,"threads_started":1}
I20260812 03:05:55.719508 31484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:55.719141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.131:46893)
0812 03:05:55.719363 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:55.719413 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":73,"threads_started":1}
I20260812 03:05:55.826313 31485 raft_consensus.cc:482] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:05:55.826478 31485 raft_consensus.cc:504] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } } peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } }
I20260812 03:05:55.826917 31485 leader_election.cc:288] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1a0f8e3c4c9b44ada0371e5a305f2bd0 (127.5.96.130:32923), 96e4c609213140e993ce7f1a08de5ec0 (127.5.96.131:45251)
W20260812 03:05:55.827785 30814 leader_election.cc:341] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1a0f8e3c4c9b44ada0371e5a305f2bd0 (127.5.96.130:32923): Not found: Tablet not found: 69ff76ea2b5f43919494e9b47b255f50
W20260812 03:05:55.827904 30820 leader_election.cc:341] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 96e4c609213140e993ce7f1a08de5ec0 (127.5.96.131:45251): Not found: Tablet not found: 69ff76ea2b5f43919494e9b47b255f50
I20260812 03:05:55.828009 30820 leader_election.cc:302] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d79d5dc5e114111b110c89a2b06456c; no voters: 1a0f8e3c4c9b44ada0371e5a305f2bd0, 96e4c609213140e993ce7f1a08de5ec0
I20260812 03:05:55.828166 31485 raft_consensus.cc:2726] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:05:56.450788 31486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:56.449994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.129:58891)
0812 03:05:56.450460 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:56.450566 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":144,"threads_started":1}
I20260812 03:05:56.533197 31487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:56.532569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.130:45655)
0812 03:05:56.532971 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:56.533061 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":133,"threads_started":1}
I20260812 03:05:56.720513 31489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:56.720156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.131:36161)
0812 03:05:56.720381 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:56.720430 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":70,"threads_started":1}
I20260812 03:05:57.452247 31490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:57.451811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.129:37315)
0812 03:05:57.452096 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:57.452146 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":76,"threads_started":1}
I20260812 03:05:57.534314 31494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:57.533941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.130:40571)
0812 03:05:57.534180 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:57.534231 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":79,"threads_started":1}
I20260812 03:05:57.721495 31496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:57.721145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.131:33337)
0812 03:05:57.721359 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:57.721408 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":70,"threads_started":1}
I20260812 03:05:57.735424 31497 raft_consensus.cc:482] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:05:57.735549 31497 raft_consensus.cc:504] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } } peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } }
I20260812 03:05:57.736058 31497 leader_election.cc:288] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1a0f8e3c4c9b44ada0371e5a305f2bd0 (127.5.96.130:32923), 96e4c609213140e993ce7f1a08de5ec0 (127.5.96.131:45251)
W20260812 03:05:57.736709 30820 leader_election.cc:341] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 96e4c609213140e993ce7f1a08de5ec0 (127.5.96.131:45251): Not found: Tablet not found: 69ff76ea2b5f43919494e9b47b255f50
W20260812 03:05:57.736814 30814 leader_election.cc:341] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1a0f8e3c4c9b44ada0371e5a305f2bd0 (127.5.96.130:32923): Not found: Tablet not found: 69ff76ea2b5f43919494e9b47b255f50
I20260812 03:05:57.736898 30814 leader_election.cc:302] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d79d5dc5e114111b110c89a2b06456c; no voters: 1a0f8e3c4c9b44ada0371e5a305f2bd0, 96e4c609213140e993ce7f1a08de5ec0
I20260812 03:05:57.737032 31497 raft_consensus.cc:2726] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:05:58.453351 31498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:58.452934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.129:49789)
0812 03:05:58.453192 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:58.453242 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":84,"threads_started":1}
I20260812 03:05:58.535281 31499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:58.534910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.130:35489)
0812 03:05:58.535137 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:58.535184 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":73,"threads_started":1}
I20260812 03:05:58.722509 31502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:58.722110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.131:37747)
0812 03:05:58.722371 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:58.722422 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":75,"threads_started":1}
I20260812 03:05:59.454416 31503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:59.453998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.129:43855)
0812 03:05:59.454266 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:59.454316 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":96,"threads_started":1}
I20260812 03:05:59.536336 31505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:59.535898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.130:53005)
0812 03:05:59.536191 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:59.536241 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":87,"threads_started":1}
I20260812 03:05:59.723564 31506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:05:59.723145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.131:53391)
0812 03:05:59.723428 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0812 03:05:59.723476 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":69,"threads_started":1}
I20260812 03:06:00.455541 31508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:00.455134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.129:54859)
0812 03:06:00.455387 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:00.455441 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":77,"threads_started":1}
I20260812 03:06:00.537343 31509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:00.536937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.130:50097)
0812 03:06:00.537185 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:00.537247 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":80,"threads_started":1}
I20260812 03:06:00.554436 31512 raft_consensus.cc:482] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:06:00.554561 31512 raft_consensus.cc:504] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } } peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } }
I20260812 03:06:00.554914 31512 leader_election.cc:288] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1a0f8e3c4c9b44ada0371e5a305f2bd0 (127.5.96.130:32923), 96e4c609213140e993ce7f1a08de5ec0 (127.5.96.131:45251)
W20260812 03:06:00.555574 30814 leader_election.cc:341] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1a0f8e3c4c9b44ada0371e5a305f2bd0 (127.5.96.130:32923): Not found: Tablet not found: 69ff76ea2b5f43919494e9b47b255f50
W20260812 03:06:00.555703 30820 leader_election.cc:341] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 96e4c609213140e993ce7f1a08de5ec0 (127.5.96.131:45251): Not found: Tablet not found: 69ff76ea2b5f43919494e9b47b255f50
I20260812 03:06:00.555786 30820 leader_election.cc:302] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d79d5dc5e114111b110c89a2b06456c; no voters: 1a0f8e3c4c9b44ada0371e5a305f2bd0, 96e4c609213140e993ce7f1a08de5ec0
I20260812 03:06:00.555938 31512 raft_consensus.cc:2726] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:06:00.724598 31513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:00.724216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.131:52633)
0812 03:06:00.724456 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:00.724507 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":71,"threads_started":1}
I20260812 03:06:01.456822 31514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:01.456330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.129:55637)
0812 03:06:01.456641 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:01.456700 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":103,"threads_started":1}
I20260812 03:06:01.538311 31515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:01.537933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.130:57065)
0812 03:06:01.538162 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:01.538213 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":72,"threads_started":1}
I20260812 03:06:01.725610 31516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:01.725211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.131:59359)
0812 03:06:01.725473 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:01.725523 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":98,"threads_started":1}
I20260812 03:06:02.458060 31517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:02.457651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.129:38499)
0812 03:06:02.457909 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:02.457960 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":108,"threads_started":1}
I20260812 03:06:02.539307 31518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:02.538924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.130:37309)
0812 03:06:02.539168 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:02.539215 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":90,"threads_started":1}
I20260812 03:06:02.726673 31519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:02.726278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.131:49663)
0812 03:06:02.726536 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:02.726585 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":89,"threads_started":1}
I20260812 03:06:03.459223 31526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:03.458818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.129:58267)
0812 03:06:03.459071 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:03.459122 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":78,"threads_started":1}
I20260812 03:06:03.540304 31528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:03.539926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.130:60513)
0812 03:06:03.540154 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:03.540204 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":69,"threads_started":1}
I20260812 03:06:03.727705 31529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:03.727292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.131:43289)
0812 03:06:03.727537 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:03.727586 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":89,"threads_started":1}
I20260812 03:06:04.460356 31530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:04.459948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.129:58253)
0812 03:06:04.460182 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:04.460237 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":76,"threads_started":1}
I20260812 03:06:04.541288 31532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:04.540909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.130:34319)
0812 03:06:04.541149 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:04.541199 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":73,"threads_started":1}
I20260812 03:06:04.728672 31533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:04.728301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.131:44367)
0812 03:06:04.728533 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:04.728585 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":68,"threads_started":1}
I20260812 03:06:05.408263 31534 raft_consensus.cc:482] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:06:05.408388 31534 raft_consensus.cc:504] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0f8e3c4c9b44ada0371e5a305f2bd0" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 32923 } } peers { permanent_uuid: "96e4c609213140e993ce7f1a08de5ec0" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45251 } } peers { permanent_uuid: "9d79d5dc5e114111b110c89a2b06456c" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34127 } }
I20260812 03:06:05.408744 31534 leader_election.cc:288] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1a0f8e3c4c9b44ada0371e5a305f2bd0 (127.5.96.130:32923), 96e4c609213140e993ce7f1a08de5ec0 (127.5.96.131:45251)
W20260812 03:06:05.409504 30814 leader_election.cc:341] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1a0f8e3c4c9b44ada0371e5a305f2bd0 (127.5.96.130:32923): Not found: Tablet not found: 69ff76ea2b5f43919494e9b47b255f50
W20260812 03:06:05.409641 30820 leader_election.cc:341] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 96e4c609213140e993ce7f1a08de5ec0 (127.5.96.131:45251): Not found: Tablet not found: 69ff76ea2b5f43919494e9b47b255f50
I20260812 03:06:05.409713 30820 leader_election.cc:302] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d79d5dc5e114111b110c89a2b06456c; no voters: 1a0f8e3c4c9b44ada0371e5a305f2bd0, 96e4c609213140e993ce7f1a08de5ec0
I20260812 03:06:05.409843 31534 raft_consensus.cc:2726] T 69ff76ea2b5f43919494e9b47b255f50 P 9d79d5dc5e114111b110c89a2b06456c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:06:05.461478 31536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:05.461059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.129:41271)
0812 03:06:05.461330 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:05.461382 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":95,"threads_started":1}
I20260812 03:06:05.542286 31538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:05.541906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.130:53973)
0812 03:06:05.542147 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:05.542196 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":92,"threads_started":1}
I20260812 03:06:05.729665 31539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:05.729294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.131:39905)
0812 03:06:05.729525 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:05.729577 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":74,"threads_started":1}
I20260812 03:06:06.462550 31541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:06.462159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.129:36967)
0812 03:06:06.462401 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:06.462454 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":72,"threads_started":1}
I20260812 03:06:06.543262 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:06.542870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.130:52169)
0812 03:06:06.543111 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:06.543160 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":73,"threads_started":1}
I20260812 03:06:06.730645 31543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:06.730281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.131:54077)
0812 03:06:06.730510 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:06.730558 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":70,"threads_started":1}
I20260812 03:06:07.463898 31544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:07.463449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.129:52895)
0812 03:06:07.463733 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:07.463798 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":76,"threads_started":1}
I20260812 03:06:07.544377 31547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:07.544020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.130:42865)
0812 03:06:07.544239 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:07.544287 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":67,"threads_started":1}
I20260812 03:06:07.718009 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 30732
I20260812 03:06:07.731986 31551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:07.731598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33641 (local address 127.5.96.131:58469)
0812 03:06:07.731847 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:07.731898 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:33641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":89,"threads_started":1}
I20260812 03:06:07.740913 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 31013
I20260812 03:06:07.756521 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 31232
2026-08-12T03:06:07Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19348 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:06:07 dist-test-slave-6bbx krb5kdc[31557](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:06:07 dist-test-slave-6bbx krb5kdc[31557](info): set up 2 sockets
Aug 12 03:06:07 dist-test-slave-6bbx krb5kdc[31557](info): commencing operation
krb5kdc: starting...
W20260812 03:06:08.922111 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:06:08 dist-test-slave-6bbx krb5kdc[31557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:06:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:06:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:06:09.032066 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:37671
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:40841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:37671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:06:09.149822 31585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:06:09.150054 31585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:06:09.150107 31585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:06:09.153937 31585 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 03:06:09.153980 31585 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 03:06:09.154000 31585 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 03:06:09.154018 31585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:06:09.154042 31585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:06:09.154073 31585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:06:09.154096 31585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:06:09.154124 31585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:06:09.159114 31585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:37671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:37671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:06:09.160053 31585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:06:09.160972 31585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:06:09.169003 31597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:06:09.168991 31590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:06:09.169181 31585 server_base.cc:1034] running on GCE node
W20260812 03:06:09.169034 31593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:06:09.169721 31585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:06:09.170719 31585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:06:09.171865 31585 hybrid_clock.cc:648] HybridClock initialized: now 1786503969171847 us; error 31 us; skew 500 ppm
Aug 12 03:06:09 dist-test-slave-6bbx krb5kdc[31557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503969, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:06:09.174356 31585 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:06:09.175146 31585 webserver.cc:460] Webserver started at http://127.5.96.190:40421/ using document root <none> and password file <none>
I20260812 03:06:09.175700 31585 fs_manager.cc:359] Metadata directory not provided
I20260812 03:06:09.175781 31585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:06:09.175981 31585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:06:09.177863 31585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "10658f3d49094703bf7640992e28bdcc"
format_stamp: "Formatted at 2026-08-12 03:06:09 on dist-test-slave-6bbx"
server_key: "a21426cbe8d38e227a147aa9ffdd3863"
server_key_iv: "c840d62ddd2f3b3a6e553caff045006b"
server_key_version: "encryptionkey@0"
I20260812 03:06:09.178399 31585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "10658f3d49094703bf7640992e28bdcc"
format_stamp: "Formatted at 2026-08-12 03:06:09 on dist-test-slave-6bbx"
server_key: "a21426cbe8d38e227a147aa9ffdd3863"
server_key_iv: "c840d62ddd2f3b3a6e553caff045006b"
server_key_version: "encryptionkey@0"
I20260812 03:06:09.181828 31585 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 03:06:09.184062 31610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:06:09.185179 31585 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:06:09.185304 31585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "10658f3d49094703bf7640992e28bdcc"
format_stamp: "Formatted at 2026-08-12 03:06:09 on dist-test-slave-6bbx"
server_key: "a21426cbe8d38e227a147aa9ffdd3863"
server_key_iv: "c840d62ddd2f3b3a6e553caff045006b"
server_key_version: "encryptionkey@0"
I20260812 03:06:09.185398 31585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:06:09.199255 31585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:06:09.202034 31585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:06:09.202210 31585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:06:09.208582 31585 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:37671
I20260812 03:06:09.208848 31697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:37671 every 8 connection(s)
I20260812 03:06:09.209443 31585 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:06:09.212023 31698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:06:09.217106 31698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc: Bootstrap starting.
I20260812 03:06:09.217388 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 31585
I20260812 03:06:09.217506 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:06:09.217787 5506 external_mini_cluster.cc:1428] Setting key 883e0ce1c2f9a408503e5083d5f71249
I20260812 03:06:09.219588 31698 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc: Neither blocks nor log segments found. Creating new log.
I20260812 03:06:09.220306 31698 log.cc:824] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc: Log is configured to *not* fsync() on all Append() calls
I20260812 03:06:09.222167 31698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc: No bootstrap required, opened a new log
Aug 12 03:06:09 dist-test-slave-6bbx krb5kdc[31557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:06:09.226318 31698 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10658f3d49094703bf7640992e28bdcc" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37671 } }
I20260812 03:06:09.226575 31698 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:06:09.226634 31698 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10658f3d49094703bf7640992e28bdcc, State: Initialized, Role: FOLLOWER
I20260812 03:06:09.227083 31698 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10658f3d49094703bf7640992e28bdcc" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37671 } }
I20260812 03:06:09.227216 31698 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:06:09.227286 31698 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:06:09.227391 31698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:06:09.228392 31698 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10658f3d49094703bf7640992e28bdcc" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37671 } }
I20260812 03:06:09.228744 31698 leader_election.cc:302] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10658f3d49094703bf7640992e28bdcc; no voters:
I20260812 03:06:09.229087 31698 leader_election.cc:288] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:06:09.229171 31709 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:06:09.229385 31709 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [term 1 LEADER]: Becoming Leader. State: Replica: 10658f3d49094703bf7640992e28bdcc, State: Running, Role: LEADER
I20260812 03:06:09.229678 31709 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10658f3d49094703bf7640992e28bdcc" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37671 } }
I20260812 03:06:09.230341 31698 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:06:09.231952 31705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:09.218987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60318 (local address 127.5.96.190:37671)
0812 03:06:09.219356 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0812 03:06:09.219371 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:06:09.219405 (+ 34us) server_negotiation.cc:407] Connection header received
0812 03:06:09.220041 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:06:09.220059 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:06:09.220373 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:06:09.220684 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:06:09.221396 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.222377 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.223519 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.224001 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.224115 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:09.226791 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:06:09.226796 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:06:09.226808 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:06:09.226833 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:06:09.228730 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:06:09.229160 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:06:09.229167 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:06:09.229173 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:06:09.229327 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:06:09.229625 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:06:09.229628 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:06:09.229630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:06:09.230148 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:06:09.230342 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 03:06:09.230897 (+ 555us) server_negotiation.cc:299] Negotiation successful
0812 03:06:09.231128 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":99,"threads_started":1}
I20260812 03:06:09.232710 31714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10658f3d49094703bf7640992e28bdcc. Latest consensus state: current_term: 1 leader_uuid: "10658f3d49094703bf7640992e28bdcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10658f3d49094703bf7640992e28bdcc" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37671 } } }
I20260812 03:06:09.233059 31714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [sys.catalog]: This master's current role is: LEADER
I20260812 03:06:09.232965 31710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10658f3d49094703bf7640992e28bdcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10658f3d49094703bf7640992e28bdcc" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37671 } } }
I20260812 03:06:09.233355 31710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [sys.catalog]: This master's current role is: LEADER
W20260812 03:06:09.235172 31736 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:06:09.235235 31736 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:06:09.235311 31738 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:06:09.236037 31738 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:06:09.241289 31738 catalog_manager.cc:1347] Generated new cluster ID: eaf0afac3404484fa868fcdba00f42be
I20260812 03:06:09.241348 31738 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:06:09.267554 31738 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:06:09.268764 31738 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:06:09.278044 31738 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc: Generated new TSK 0
I20260812 03:06:09.278740 31738 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.96.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
I20260812 03:06:09.308187 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:0
--local_ip_for_outbound_sockets=127.5.96.129
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:37671
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:40841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:06:09.415194 31744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:06:09.415409 31744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:06:09.415448 31744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:06:09.419265 31744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:06:09.419312 31744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:06:09.419380 31744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:06:09.424340 31744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:37671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:06:09.425239 31744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:06:09.426168 31744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:06:09.435359 31753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:06:09.435389 31756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:06:09.435346 31752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:06:09.435582 31744 server_base.cc:1034] running on GCE node
I20260812 03:06:09.436076 31744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:06:09.436643 31744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:06:09.437808 31744 hybrid_clock.cc:648] HybridClock initialized: now 1786503969437794 us; error 23 us; skew 500 ppm
Aug 12 03:06:09 dist-test-slave-6bbx krb5kdc[31557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503969, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:06:09.440335 31744 init.cc:377] Logged in from keytab as kudu/127.5.96.129@KRBTEST.COM (short username kudu)
I20260812 03:06:09.441123 31744 webserver.cc:460] Webserver started at http://127.5.96.129:43481/ using document root <none> and password file <none>
I20260812 03:06:09.441625 31744 fs_manager.cc:359] Metadata directory not provided
I20260812 03:06:09.441690 31744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:06:09.441879 31744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:06:09.443845 31744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/data/instance:
uuid: "cfcd528c8f4b40b5a86d56d912041895"
format_stamp: "Formatted at 2026-08-12 03:06:09 on dist-test-slave-6bbx"
server_key: "c3ae28dcaf27d913bc52b134a194cb81"
server_key_iv: "9636a49d4125b56badd25a674f9b3575"
server_key_version: "encryptionkey@0"
I20260812 03:06:09.444367 31744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance:
uuid: "cfcd528c8f4b40b5a86d56d912041895"
format_stamp: "Formatted at 2026-08-12 03:06:09 on dist-test-slave-6bbx"
server_key: "c3ae28dcaf27d913bc52b134a194cb81"
server_key_iv: "9636a49d4125b56badd25a674f9b3575"
server_key_version: "encryptionkey@0"
I20260812 03:06:09.447847 31744 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:06:09.450002 31766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:06:09.451100 31744 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:06:09.451213 31744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "cfcd528c8f4b40b5a86d56d912041895"
format_stamp: "Formatted at 2026-08-12 03:06:09 on dist-test-slave-6bbx"
server_key: "c3ae28dcaf27d913bc52b134a194cb81"
server_key_iv: "9636a49d4125b56badd25a674f9b3575"
server_key_version: "encryptionkey@0"
I20260812 03:06:09.451305 31744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:06:09.470232 31744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:06:09.472769 31744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:06:09.472926 31744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:06:09.473433 31744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:06:09.474188 31744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:06:09.474234 31744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:06:09.474292 31744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:06:09.474321 31744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:06:09.482278 31744 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:36745
I20260812 03:06:09.482331 31951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:36745 every 8 connection(s)
I20260812 03:06:09.483071 31744 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:06:09.483608 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 31744
I20260812 03:06:09.483794 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance
I20260812 03:06:09.484119 5506 external_mini_cluster.cc:1428] Setting key e98402f6850df33996789b1e8bbee1ab
Aug 12 03:06:09 dist-test-slave-6bbx krb5kdc[31557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503969, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.96.130@KRBTEST.COM; defaulting to no policy
I20260812 03:06:09.496495 31705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:09.484857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:55439 (local address 127.5.96.190:37671)
0812 03:06:09.484960 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 03:06:09.484964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:06:09.485921 (+ 957us) server_negotiation.cc:407] Connection header received
0812 03:06:09.487174 (+ 1253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:06:09.487177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:06:09.487242 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:06:09.487336 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:06:09.488787 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.489372 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.490463 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.490695 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.490789 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:09.493200 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:06:09.493203 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:06:09.493205 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:06:09.493229 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:06:09.494493 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:06:09.495049 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:06:09.495052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:06:09.495053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:06:09.495128 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:06:09.495426 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:06:09.495429 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:06:09.495430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:06:09.495586 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:06:09.495655 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:06:09.496184 (+ 529us) server_negotiation.cc:299] Negotiation successful
0812 03:06:09.496324 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.130@KRBTEST.COM" created.
I20260812 03:06:09.497265 31954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:09.485302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.129:55439)
0812 03:06:09.485766 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:09.485817 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 03:06:09.486978 (+ 1161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:06:09.487532 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:06:09.487541 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:06:09.488017 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:06:09.488634 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:09.488652 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.489485 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.489488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:09.490341 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:09.490350 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.490847 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.490851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:09.491006 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:09.491491 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:06:09.491507 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:06:09.493019 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:06:09.494618 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:06:09.494625 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:06:09.494638 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:06:09.494962 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:06:09.495200 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:06:09.495202 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:06:09.495204 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:06:09.495338 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:06:09.495710 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:06:09.495716 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:06:09.495965 (+ 249us) client_negotiation.cc:769] Sending connection context
0812 03:06:09.496117 (+ 152us) client_negotiation.cc:240] Negotiation successful
0812 03:06:09.496414 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":118,"threads_started":1}
I20260812 03:06:09.499579 31952 heartbeater.cc:347] Connected to a master server at 127.5.96.190:37671
I20260812 03:06:09.499828 31952 heartbeater.cc:464] Registering TS with master...
I20260812 03:06:09.500249 31952 heartbeater.cc:511] Master 127.5.96.190:37671 requested a full tablet report, sending...
I20260812 03:06:09.501607 31638 ts_manager.cc:194] Registered new tserver with Master: cfcd528c8f4b40b5a86d56d912041895 (127.5.96.129:36745)
I20260812 03:06:09.502722 31638 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.129@KRBTEST.COM'} at 127.5.96.129:55439
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
I20260812 03:06:09.512019 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:0
--local_ip_for_outbound_sockets=127.5.96.130
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:37671
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:40841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:06:09.620507 31958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:06:09.620739 31958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:06:09.620795 31958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:06:09.624678 31958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:06:09.624734 31958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:06:09.624805 31958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:06:09.629736 31958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:37671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:06:09.630609 31958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:06:09.631577 31958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:06:09.640445 31968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:06:09.640514 31965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:06:09.640542 31964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:06:09.640690 31958 server_base.cc:1034] running on GCE node
I20260812 03:06:09.641101 31958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:06:09.641695 31958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:06:09.642859 31958 hybrid_clock.cc:648] HybridClock initialized: now 1786503969642842 us; error 28 us; skew 500 ppm
Aug 12 03:06:09 dist-test-slave-6bbx krb5kdc[31557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503969, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:06:09.645329 31958 init.cc:377] Logged in from keytab as kudu/127.5.96.130@KRBTEST.COM (short username kudu)
I20260812 03:06:09.646113 31958 webserver.cc:460] Webserver started at http://127.5.96.130:39199/ using document root <none> and password file <none>
I20260812 03:06:09.646600 31958 fs_manager.cc:359] Metadata directory not provided
I20260812 03:06:09.646663 31958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:06:09.646847 31958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:06:09.648779 31958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/data/instance:
uuid: "84410a7f07064d2eb11893a02950535f"
format_stamp: "Formatted at 2026-08-12 03:06:09 on dist-test-slave-6bbx"
server_key: "64c6f023da0804f0b7c2192356efc641"
server_key_iv: "7546d210ba91b0581c39fbba40607246"
server_key_version: "encryptionkey@0"
I20260812 03:06:09.649291 31958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance:
uuid: "84410a7f07064d2eb11893a02950535f"
format_stamp: "Formatted at 2026-08-12 03:06:09 on dist-test-slave-6bbx"
server_key: "64c6f023da0804f0b7c2192356efc641"
server_key_iv: "7546d210ba91b0581c39fbba40607246"
server_key_version: "encryptionkey@0"
I20260812 03:06:09.652647 31958 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:06:09.654784 31977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:06:09.655939 31958 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:06:09.656041 31958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "84410a7f07064d2eb11893a02950535f"
format_stamp: "Formatted at 2026-08-12 03:06:09 on dist-test-slave-6bbx"
server_key: "64c6f023da0804f0b7c2192356efc641"
server_key_iv: "7546d210ba91b0581c39fbba40607246"
server_key_version: "encryptionkey@0"
I20260812 03:06:09.656154 31958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:06:09.674096 31958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:06:09.676703 31958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:06:09.676851 31958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:06:09.677343 31958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:06:09.678118 31958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:06:09.678169 31958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:06:09.678205 31958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:06:09.678226 31958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:06:09.686575 31958 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:40785
I20260812 03:06:09.686604 32148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:40785 every 8 connection(s)
I20260812 03:06:09.687381 31958 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:06:09.687800 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 31958
I20260812 03:06:09.687923 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance
I20260812 03:06:09.688220 5506 external_mini_cluster.cc:1428] Setting key 4eecda09f0222eda9de833097cc5ec6b
Aug 12 03:06:09 dist-test-slave-6bbx krb5kdc[31557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503969, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.96.131@KRBTEST.COM; defaulting to no policy
I20260812 03:06:09.701650 31705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:09.689171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:42825 (local address 127.5.96.190:37671)
0812 03:06:09.689309 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 03:06:09.689314 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:06:09.690304 (+ 990us) server_negotiation.cc:407] Connection header received
0812 03:06:09.691843 (+ 1539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:06:09.691846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:06:09.691901 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:06:09.691989 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:06:09.693097 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.693738 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.695356 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.695579 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.695652 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:09.698401 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:06:09.698404 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:06:09.698406 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:06:09.698431 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:06:09.699785 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:06:09.700212 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:06:09.700214 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:06:09.700215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:06:09.700281 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:06:09.700609 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:06:09.700612 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:06:09.700614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:06:09.700799 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:06:09.700884 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 03:06:09.701364 (+ 480us) server_negotiation.cc:299] Negotiation successful
0812 03:06:09.701487 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.131@KRBTEST.COM" created.
I20260812 03:06:09.702504 32151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:09.689613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.130:42825)
0812 03:06:09.690134 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:09.690190 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 03:06:09.691622 (+ 1432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:06:09.692158 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:06:09.692165 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:06:09.692498 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:06:09.692950 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:09.692963 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.693875 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.693879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:09.695242 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:09.695254 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.695749 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.695753 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:09.695968 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:09.696534 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:06:09.696556 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:06:09.698233 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:06:09.699880 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:06:09.699885 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:06:09.699896 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:06:09.700138 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:06:09.700373 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:06:09.700376 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:06:09.700379 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:06:09.700513 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:06:09.700903 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:06:09.700909 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:06:09.701127 (+ 218us) client_negotiation.cc:769] Sending connection context
0812 03:06:09.701293 (+ 166us) client_negotiation.cc:240] Negotiation successful
0812 03:06:09.701592 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":127,"threads_started":1}
I20260812 03:06:09.703689 32149 heartbeater.cc:347] Connected to a master server at 127.5.96.190:37671
I20260812 03:06:09.703943 32149 heartbeater.cc:464] Registering TS with master...
I20260812 03:06:09.704509 32149 heartbeater.cc:511] Master 127.5.96.190:37671 requested a full tablet report, sending...
I20260812 03:06:09.705605 31638 ts_manager.cc:194] Registered new tserver with Master: 84410a7f07064d2eb11893a02950535f (127.5.96.130:40785)
I20260812 03:06:09.706144 31638 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.130@KRBTEST.COM'} at 127.5.96.130:42825
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
I20260812 03:06:09.716173 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:0
--local_ip_for_outbound_sockets=127.5.96.131
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:37671
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:40841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:06:09.823391 32154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:06:09.823622 32154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:06:09.823678 32154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:06:09.827548 32154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:06:09.827600 32154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:06:09.827656 32154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:06:09.832854 32154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:37671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:06:09.833853 32154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:06:09.834825 32154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:06:09.844193 32164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:06:09.844202 32167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:06:09.844280 32154 server_base.cc:1034] running on GCE node
W20260812 03:06:09.844193 32161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:06:09.844786 32154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:06:09.845371 32154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:06:09.846519 32154 hybrid_clock.cc:648] HybridClock initialized: now 1786503969846498 us; error 31 us; skew 500 ppm
Aug 12 03:06:09 dist-test-slave-6bbx krb5kdc[31557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503969, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:06:09.849215 32154 init.cc:377] Logged in from keytab as kudu/127.5.96.131@KRBTEST.COM (short username kudu)
I20260812 03:06:09.850071 32154 webserver.cc:460] Webserver started at http://127.5.96.131:33305/ using document root <none> and password file <none>
I20260812 03:06:09.850589 32154 fs_manager.cc:359] Metadata directory not provided
I20260812 03:06:09.850658 32154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:06:09.850855 32154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:06:09.852823 32154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/data/instance:
uuid: "ce9c4afdaee0491ba04d19e2b31fd28f"
format_stamp: "Formatted at 2026-08-12 03:06:09 on dist-test-slave-6bbx"
server_key: "0cb1831e4bbf339c6ed44118c874d5ca"
server_key_iv: "50952497d09213753ac87783dec2c35a"
server_key_version: "encryptionkey@0"
I20260812 03:06:09.853345 32154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance:
uuid: "ce9c4afdaee0491ba04d19e2b31fd28f"
format_stamp: "Formatted at 2026-08-12 03:06:09 on dist-test-slave-6bbx"
server_key: "0cb1831e4bbf339c6ed44118c874d5ca"
server_key_iv: "50952497d09213753ac87783dec2c35a"
server_key_version: "encryptionkey@0"
I20260812 03:06:09.857064 32154 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:06:09.859390 32177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:06:09.860833 32154 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 03:06:09.860939 32154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "ce9c4afdaee0491ba04d19e2b31fd28f"
format_stamp: "Formatted at 2026-08-12 03:06:09 on dist-test-slave-6bbx"
server_key: "0cb1831e4bbf339c6ed44118c874d5ca"
server_key_iv: "50952497d09213753ac87783dec2c35a"
server_key_version: "encryptionkey@0"
I20260812 03:06:09.861032 32154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:06:09.872560 32154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:06:09.875474 32154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:06:09.875741 32154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:06:09.876299 32154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:06:09.877095 32154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:06:09.877142 32154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:06:09.877194 32154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:06:09.877224 32154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:06:09.885699 32154 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:36495
I20260812 03:06:09.885727 32357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:36495 every 8 connection(s)
I20260812 03:06:09.886533 32154 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:06:09.891584 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 32154
I20260812 03:06:09.891716 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance
I20260812 03:06:09.891976 5506 external_mini_cluster.cc:1428] Setting key 269ba934619519b644fe6b32e25effe0
Aug 12 03:06:09 dist-test-slave-6bbx krb5kdc[31557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503969, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:06:09.899168 31705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:09.888227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:53673 (local address 127.5.96.190:37671)
0812 03:06:09.888370 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 03:06:09.888374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:06:09.889156 (+ 782us) server_negotiation.cc:407] Connection header received
0812 03:06:09.890453 (+ 1297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:06:09.890456 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:06:09.890503 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:06:09.890573 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:06:09.891716 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.892373 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.893763 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.894045 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.894126 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:09.896414 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:06:09.896417 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:06:09.896419 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:06:09.896449 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:06:09.897615 (+ 1166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:06:09.898037 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:06:09.898039 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:06:09.898041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:06:09.898106 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:06:09.898359 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:06:09.898361 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:06:09.898362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:06:09.898517 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:06:09.898584 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:06:09.898967 (+ 383us) server_negotiation.cc:299] Negotiation successful
0812 03:06:09.899054 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 03:06:09.899943 32360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:09.888548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.131:53673)
0812 03:06:09.889003 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:09.889046 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:06:09.890264 (+ 1218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:06:09.890733 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:06:09.890739 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:06:09.891078 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:06:09.891538 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:09.891551 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.892504 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.892507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:09.893631 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:09.893641 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.894173 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.894176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:09.894314 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:09.894834 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:06:09.894849 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:06:09.896246 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:06:09.897713 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:06:09.897718 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:06:09.897728 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:06:09.897966 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:06:09.898167 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:06:09.898169 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:06:09.898171 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:06:09.898297 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:06:09.898584 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:06:09.898588 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:06:09.898793 (+ 205us) client_negotiation.cc:769] Sending connection context
0812 03:06:09.898933 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 03:06:09.899155 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":119,"threads_started":1}
I20260812 03:06:09.900952 32358 heartbeater.cc:347] Connected to a master server at 127.5.96.190:37671
I20260812 03:06:09.901146 32358 heartbeater.cc:464] Registering TS with master...
I20260812 03:06:09.901609 32358 heartbeater.cc:511] Master 127.5.96.190:37671 requested a full tablet report, sending...
I20260812 03:06:09.902667 31638 ts_manager.cc:194] Registered new tserver with Master: ce9c4afdaee0491ba04d19e2b31fd28f (127.5.96.131:36495)
I20260812 03:06:09.903183 31638 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.131@KRBTEST.COM'} at 127.5.96.131:53673
I20260812 03:06:09.905403 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:06:09.913618 31705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:09.907188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60334 (local address 127.5.96.190:37671)
0812 03:06:09.907302 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 03:06:09.907305 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:06:09.907732 (+ 427us) server_negotiation.cc:407] Connection header received
0812 03:06:09.907848 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:06:09.907851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:06:09.907889 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:06:09.907955 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:06:09.908518 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.908988 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.909735 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.909933 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.910002 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:09.911158 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:06:09.911160 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:06:09.911162 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:06:09.911184 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:06:09.912334 (+ 1150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:06:09.912674 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:06:09.912676 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:06:09.912677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:06:09.912742 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:06:09.912923 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:06:09.912924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:06:09.912925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:06:09.913133 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:06:09.913234 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 03:06:09.913388 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 03:06:09.913478 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:06:09.916131 31638 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60334:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:06:09.918934 31638 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:06:09.932744 32373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:09.926917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:40785 (local address 127.0.0.1:53438)
0812 03:06:09.927209 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:09.927252 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:06:09.927370 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:06:09.928346 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:06:09.928352 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:06:09.928376 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:06:09.928496 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:09.928504 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.929860 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.929863 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:09.931031 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:09.931041 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.932234 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.932237 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:09.932351 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:09.932387 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 03:06:09.932493 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 03:06:09.932607 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":1,"thread_start_us":68,"threads_started":1}
I20260812 03:06:09.932765 32377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:09.927065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53438 (local address 127.5.96.130:40785)
0812 03:06:09.927941 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0812 03:06:09.927953 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:06:09.927976 (+ 23us) server_negotiation.cc:407] Connection header received
0812 03:06:09.928040 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:06:09.928043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:06:09.928194 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:06:09.928329 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:06:09.928634 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.929705 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.931161 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.932114 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.932215 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:09.932362 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 03:06:09.932534 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 03:06:09.932625 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":770,"thread_start_us":141,"threads_started":1}
I20260812 03:06:09.933219 32378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:09.927723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:36745 (local address 127.0.0.1:44828)
0812 03:06:09.927972 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:09.927986 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:06:09.928075 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:06:09.929079 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:06:09.929082 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:06:09.929101 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:06:09.929195 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:09.929200 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.930513 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.930515 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:09.931770 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:09.931777 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.932875 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.932879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:09.932976 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:09.932998 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:06:09.933047 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 03:06:09.933116 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":26,"thread_start_us":103,"threads_started":1}
I20260812 03:06:09.933462 32376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:09.926991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:36495 (local address 127.0.0.1:52990)
0812 03:06:09.927568 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:09.927580 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:06:09.927694 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:06:09.929117 (+ 1423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:06:09.929120 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:06:09.929150 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:06:09.929237 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:09.929243 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.930433 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.930437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:09.931756 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:09.931768 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.932875 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.932879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:09.933156 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:09.933182 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 03:06:09.933265 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 03:06:09.933335 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"mutex_wait_us":179,"spinlock_wait_cycles":896,"thread_start_us":94,"threads_started":1}
I20260812 03:06:09.933568 32380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:09.927697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52990 (local address 127.5.96.131:36495)
0812 03:06:09.928670 (+ 973us) server_negotiation.cc:206] Beginning negotiation
0812 03:06:09.928685 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:06:09.928705 (+ 20us) server_negotiation.cc:407] Connection header received
0812 03:06:09.928769 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:06:09.928773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:06:09.928947 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:06:09.929378 (+ 431us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:06:09.929506 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.930294 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.931906 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.932730 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.932924 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:09.933081 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 03:06:09.933334 (+ 253us) server_negotiation.cc:299] Negotiation successful
0812 03:06:09.933431 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":863,"thread_start_us":148,"threads_started":1}
I20260812 03:06:09.934176 32381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:09.928093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44828 (local address 127.5.96.129:36745)
0812 03:06:09.928441 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0812 03:06:09.928451 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:06:09.928473 (+ 22us) server_negotiation.cc:407] Connection header received
0812 03:06:09.928548 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:06:09.928553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:06:09.928935 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:06:09.929316 (+ 381us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:06:09.929443 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.930336 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.931903 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.932730 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.933739 (+ 1009us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:09.933896 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 03:06:09.933946 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 03:06:09.934037 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":146,"threads_started":1}
I20260812 03:06:09.935894 32050 tablet_service.cc:1467] Processing CreateTablet for tablet dc50788aaa40455bb90570583ea51ff5 (DEFAULT_TABLE table=test-table [id=763c59dfd2a146eca8f39f3cf72bd25f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:06:09.936323 32258 tablet_service.cc:1467] Processing CreateTablet for tablet dc50788aaa40455bb90570583ea51ff5 (DEFAULT_TABLE table=test-table [id=763c59dfd2a146eca8f39f3cf72bd25f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:06:09.937042 32050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc50788aaa40455bb90570583ea51ff5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:06:09.937220 32258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc50788aaa40455bb90570583ea51ff5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:06:09.936848 31842 tablet_service.cc:1467] Processing CreateTablet for tablet dc50788aaa40455bb90570583ea51ff5 (DEFAULT_TABLE table=test-table [id=763c59dfd2a146eca8f39f3cf72bd25f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:06:09.937767 31842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc50788aaa40455bb90570583ea51ff5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:06:09.943130 32382 tablet_bootstrap.cc:492] T dc50788aaa40455bb90570583ea51ff5 P ce9c4afdaee0491ba04d19e2b31fd28f: Bootstrap starting.
I20260812 03:06:09.944456 32384 tablet_bootstrap.cc:492] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895: Bootstrap starting.
I20260812 03:06:09.944512 32383 tablet_bootstrap.cc:492] T dc50788aaa40455bb90570583ea51ff5 P 84410a7f07064d2eb11893a02950535f: Bootstrap starting.
I20260812 03:06:09.946256 32382 tablet_bootstrap.cc:654] T dc50788aaa40455bb90570583ea51ff5 P ce9c4afdaee0491ba04d19e2b31fd28f: Neither blocks nor log segments found. Creating new log.
I20260812 03:06:09.946646 32383 tablet_bootstrap.cc:654] T dc50788aaa40455bb90570583ea51ff5 P 84410a7f07064d2eb11893a02950535f: Neither blocks nor log segments found. Creating new log.
I20260812 03:06:09.946938 32384 tablet_bootstrap.cc:654] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895: Neither blocks nor log segments found. Creating new log.
I20260812 03:06:09.947358 32383 log.cc:824] T dc50788aaa40455bb90570583ea51ff5 P 84410a7f07064d2eb11893a02950535f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:06:09.947495 32382 log.cc:824] T dc50788aaa40455bb90570583ea51ff5 P ce9c4afdaee0491ba04d19e2b31fd28f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:06:09.947705 32384 log.cc:824] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895: Log is configured to *not* fsync() on all Append() calls
I20260812 03:06:09.949285 32382 tablet_bootstrap.cc:492] T dc50788aaa40455bb90570583ea51ff5 P ce9c4afdaee0491ba04d19e2b31fd28f: No bootstrap required, opened a new log
I20260812 03:06:09.949363 32383 tablet_bootstrap.cc:492] T dc50788aaa40455bb90570583ea51ff5 P 84410a7f07064d2eb11893a02950535f: No bootstrap required, opened a new log
I20260812 03:06:09.949433 32384 tablet_bootstrap.cc:492] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895: No bootstrap required, opened a new log
I20260812 03:06:09.949481 32382 ts_tablet_manager.cc:1397] T dc50788aaa40455bb90570583ea51ff5 P ce9c4afdaee0491ba04d19e2b31fd28f: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 03:06:09.949532 32383 ts_tablet_manager.cc:1397] T dc50788aaa40455bb90570583ea51ff5 P 84410a7f07064d2eb11893a02950535f: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260812 03:06:09.949560 32384 ts_tablet_manager.cc:1397] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 03:06:09.952412 32382 raft_consensus.cc:348] T dc50788aaa40455bb90570583ea51ff5 P ce9c4afdaee0491ba04d19e2b31fd28f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } }
I20260812 03:06:09.952632 32382 raft_consensus.cc:374] T dc50788aaa40455bb90570583ea51ff5 P ce9c4afdaee0491ba04d19e2b31fd28f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:06:09.952703 32382 raft_consensus.cc:729] T dc50788aaa40455bb90570583ea51ff5 P ce9c4afdaee0491ba04d19e2b31fd28f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce9c4afdaee0491ba04d19e2b31fd28f, State: Initialized, Role: FOLLOWER
I20260812 03:06:09.953135 32382 consensus_queue.cc:260] T dc50788aaa40455bb90570583ea51ff5 P ce9c4afdaee0491ba04d19e2b31fd28f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } }
I20260812 03:06:09.953212 32384 raft_consensus.cc:348] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } }
I20260812 03:06:09.953212 32383 raft_consensus.cc:348] T dc50788aaa40455bb90570583ea51ff5 P 84410a7f07064d2eb11893a02950535f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } }
I20260812 03:06:09.953455 32383 raft_consensus.cc:374] T dc50788aaa40455bb90570583ea51ff5 P 84410a7f07064d2eb11893a02950535f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:06:09.953495 32384 raft_consensus.cc:374] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:06:09.953572 32384 raft_consensus.cc:729] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfcd528c8f4b40b5a86d56d912041895, State: Initialized, Role: FOLLOWER
I20260812 03:06:09.953583 32383 raft_consensus.cc:729] T dc50788aaa40455bb90570583ea51ff5 P 84410a7f07064d2eb11893a02950535f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84410a7f07064d2eb11893a02950535f, State: Initialized, Role: FOLLOWER
I20260812 03:06:09.953778 32358 heartbeater.cc:503] Master 127.5.96.190:37671 was elected leader, sending a full tablet report...
I20260812 03:06:09.954068 32382 ts_tablet_manager.cc:1428] T dc50788aaa40455bb90570583ea51ff5 P ce9c4afdaee0491ba04d19e2b31fd28f: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260812 03:06:09.954037 32383 consensus_queue.cc:260] T dc50788aaa40455bb90570583ea51ff5 P 84410a7f07064d2eb11893a02950535f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } }
I20260812 03:06:09.954037 32384 consensus_queue.cc:260] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } }
I20260812 03:06:09.954893 32149 heartbeater.cc:503] Master 127.5.96.190:37671 was elected leader, sending a full tablet report...
I20260812 03:06:09.954941 32384 ts_tablet_manager.cc:1428] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 03:06:09.954975 32391 raft_consensus.cc:482] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:06:09.955255 31952 heartbeater.cc:503] Master 127.5.96.190:37671 was elected leader, sending a full tablet report...
I20260812 03:06:09.955499 32391 raft_consensus.cc:504] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } }
I20260812 03:06:09.955989 32383 ts_tablet_manager.cc:1428] T dc50788aaa40455bb90570583ea51ff5 P 84410a7f07064d2eb11893a02950535f: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260812 03:06:09.956743 32391 leader_election.cc:288] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84410a7f07064d2eb11893a02950535f (127.5.96.130:40785), ce9c4afdaee0491ba04d19e2b31fd28f (127.5.96.131:36495)
I20260812 03:06:09.961036 31954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:09.956884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:40785 (local address 127.5.96.129:58075)
0812 03:06:09.956989 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:09.957004 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:06:09.957076 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:06:09.957439 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:06:09.957442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:06:09.957459 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:06:09.957579 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:09.957587 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.958533 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.958536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:09.959865 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:09.959874 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.960644 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.960647 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:09.960749 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:09.960771 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:06:09.960839 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 03:06:09.960907 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":30,"spinlock_wait_cycles":13568}
I20260812 03:06:09.961442 32377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:09.957028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:58075 (local address 127.5.96.130:40785)
0812 03:06:09.957138 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 03:06:09.957142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:06:09.957156 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:06:09.957198 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:06:09.957202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:06:09.957243 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:06:09.957328 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:06:09.957705 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.958403 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.959993 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.960535 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.960644 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:09.960694 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 03:06:09.961269 (+ 575us) server_negotiation.cc:299] Negotiation successful
0812 03:06:09.961340 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 03:06:09.961472 32380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:09.957203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:44251 (local address 127.5.96.131:36495)
0812 03:06:09.957298 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 03:06:09.957302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:06:09.957426 (+ 124us) server_negotiation.cc:407] Connection header received
0812 03:06:09.957617 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:06:09.957621 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:06:09.957667 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:06:09.957714 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:06:09.958117 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.958752 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.960254 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.960809 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.960889 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:09.960934 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 03:06:09.961319 (+ 385us) server_negotiation.cc:299] Negotiation successful
0812 03:06:09.961381 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:06:09.961519 32392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:09.957113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:36495 (local address 127.5.96.129:44251)
0812 03:06:09.957351 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:09.957363 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:06:09.957502 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:06:09.957851 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:06:09.957854 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:06:09.957871 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:06:09.957981 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:09.957988 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.958886 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.958889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:09.960124 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:09.960135 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.961028 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.961031 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:09.961157 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:09.961184 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 03:06:09.961299 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 03:06:09.961375 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":14,"thread_start_us":87,"threads_started":1}
I20260812 03:06:09.961948 32079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc50788aaa40455bb90570583ea51ff5" candidate_uuid: "cfcd528c8f4b40b5a86d56d912041895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84410a7f07064d2eb11893a02950535f" is_pre_election: true
I20260812 03:06:09.962105 32291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc50788aaa40455bb90570583ea51ff5" candidate_uuid: "cfcd528c8f4b40b5a86d56d912041895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" is_pre_election: true
I20260812 03:06:09.962177 32079 raft_consensus.cc:2454] T dc50788aaa40455bb90570583ea51ff5 P 84410a7f07064d2eb11893a02950535f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfcd528c8f4b40b5a86d56d912041895 in term 0.
I20260812 03:06:09.962306 32291 raft_consensus.cc:2454] T dc50788aaa40455bb90570583ea51ff5 P ce9c4afdaee0491ba04d19e2b31fd28f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfcd528c8f4b40b5a86d56d912041895 in term 0.
I20260812 03:06:09.962613 31767 leader_election.cc:302] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84410a7f07064d2eb11893a02950535f, cfcd528c8f4b40b5a86d56d912041895; no voters:
I20260812 03:06:09.962854 32391 raft_consensus.cc:2781] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:06:09.962945 32391 raft_consensus.cc:482] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:06:09.962999 32391 raft_consensus.cc:3037] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:06:09.963888 32391 raft_consensus.cc:504] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } }
I20260812 03:06:09.964205 32391 leader_election.cc:288] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895 [CANDIDATE]: Term 1 election: Requested vote from peers 84410a7f07064d2eb11893a02950535f (127.5.96.130:40785), ce9c4afdaee0491ba04d19e2b31fd28f (127.5.96.131:36495)
I20260812 03:06:09.964516 32079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc50788aaa40455bb90570583ea51ff5" candidate_uuid: "cfcd528c8f4b40b5a86d56d912041895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84410a7f07064d2eb11893a02950535f"
I20260812 03:06:09.964524 32291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc50788aaa40455bb90570583ea51ff5" candidate_uuid: "cfcd528c8f4b40b5a86d56d912041895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f"
I20260812 03:06:09.964613 32079 raft_consensus.cc:3037] T dc50788aaa40455bb90570583ea51ff5 P 84410a7f07064d2eb11893a02950535f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:06:09.964648 32291 raft_consensus.cc:3037] T dc50788aaa40455bb90570583ea51ff5 P ce9c4afdaee0491ba04d19e2b31fd28f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:06:09.965520 32291 raft_consensus.cc:2454] T dc50788aaa40455bb90570583ea51ff5 P ce9c4afdaee0491ba04d19e2b31fd28f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfcd528c8f4b40b5a86d56d912041895 in term 1.
I20260812 03:06:09.965546 32079 raft_consensus.cc:2454] T dc50788aaa40455bb90570583ea51ff5 P 84410a7f07064d2eb11893a02950535f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfcd528c8f4b40b5a86d56d912041895 in term 1.
I20260812 03:06:09.965824 31767 leader_election.cc:302] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84410a7f07064d2eb11893a02950535f, cfcd528c8f4b40b5a86d56d912041895; no voters:
I20260812 03:06:09.966006 32391 raft_consensus.cc:2781] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:06:09.966109 32391 raft_consensus.cc:686] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895 [term 1 LEADER]: Becoming Leader. State: Replica: cfcd528c8f4b40b5a86d56d912041895, State: Running, Role: LEADER
I20260812 03:06:09.966382 32391 consensus_queue.cc:237] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } }
I20260812 03:06:09.969202 31638 catalog_manager.cc:5495] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895 reported cstate change: term changed from 0 to 1, leader changed from <none> to cfcd528c8f4b40b5a86d56d912041895 (127.5.96.129). New cstate: current_term: 1 leader_uuid: "cfcd528c8f4b40b5a86d56d912041895" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } health_report { overall_health: HEALTHY } } }
W20260812 03:06:09.984103 31953 tablet.cc:2369] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:06:09.985368 32381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:09.981981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44834 (local address 127.5.96.129:36745)
0812 03:06:09.982115 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 03:06:09.982119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:06:09.982132 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:06:09.982275 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:06:09.982277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:06:09.982327 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:06:09.982412 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:06:09.982657 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.983135 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.984037 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:09.984274 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:09.984364 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:09.984655 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:06:09.985013 (+ 358us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:06:09.985115 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 03:06:09.985191 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 03:06:09.985248 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 03:06:09.990509 32079 raft_consensus.cc:1261] T dc50788aaa40455bb90570583ea51ff5 P 84410a7f07064d2eb11893a02950535f [term 1 FOLLOWER]: Refusing update from remote peer cfcd528c8f4b40b5a86d56d912041895: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:06:09.990511 32291 raft_consensus.cc:1261] T dc50788aaa40455bb90570583ea51ff5 P ce9c4afdaee0491ba04d19e2b31fd28f [term 1 FOLLOWER]: Refusing update from remote peer cfcd528c8f4b40b5a86d56d912041895: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:06:09.991147 32389 consensus_queue.cc:1035] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:06:09.991314 32391 consensus_queue.cc:1035] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:06:09.998158 32397 mvcc.cc:205] Tried to move back new op lower bound from 7317520261074636800 to 7317520260984725504. Current Snapshot: MvccSnapshot[applied={T|T < 7317520261074636800}]
I20260812 03:06:09.998328 32395 mvcc.cc:205] Tried to move back new op lower bound from 7317520261074636800 to 7317520260984725504. Current Snapshot: MvccSnapshot[applied={T|T < 7317520261074636800}]
I20260812 03:06:10.000208 32396 mvcc.cc:205] Tried to move back new op lower bound from 7317520261074636800 to 7317520260984725504. Current Snapshot: MvccSnapshot[applied={T|T < 7317520261074636800}]
I20260812 03:06:10.006305 31635 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:06:10.006445 31635 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:06:10.008692 31635 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc: Sending DeleteTablet for 3 replicas of tablet dc50788aaa40455bb90570583ea51ff5
I20260812 03:06:10.009349 32050 tablet_service.cc:1514] Processing DeleteTablet for tablet dc50788aaa40455bb90570583ea51ff5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:06:10 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:53438
I20260812 03:06:10.009560 31842 tablet_service.cc:1514] Processing DeleteTablet for tablet dc50788aaa40455bb90570583ea51ff5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:06:10 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:44828
I20260812 03:06:10.009644 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 31744
I20260812 03:06:10.009811 32258 tablet_service.cc:1514] Processing DeleteTablet for tablet dc50788aaa40455bb90570583ea51ff5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:06:10 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:52990
I20260812 03:06:10.016673 32413 tablet_replica.cc:330] T dc50788aaa40455bb90570583ea51ff5 P ce9c4afdaee0491ba04d19e2b31fd28f: stopping tablet replica
I20260812 03:06:10.016983 32413 raft_consensus.cc:2229] T dc50788aaa40455bb90570583ea51ff5 P ce9c4afdaee0491ba04d19e2b31fd28f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:06:10.017267 32413 raft_consensus.cc:2258] T dc50788aaa40455bb90570583ea51ff5 P ce9c4afdaee0491ba04d19e2b31fd28f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:06:10.019758 32412 tablet_replica.cc:330] T dc50788aaa40455bb90570583ea51ff5 P 84410a7f07064d2eb11893a02950535f: stopping tablet replica
I20260812 03:06:10.020123 32412 raft_consensus.cc:2229] T dc50788aaa40455bb90570583ea51ff5 P 84410a7f07064d2eb11893a02950535f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:06:10.020418 32412 raft_consensus.cc:2258] T dc50788aaa40455bb90570583ea51ff5 P 84410a7f07064d2eb11893a02950535f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:06:10.020442 32413 ts_tablet_manager.cc:1905] T dc50788aaa40455bb90570583ea51ff5 P ce9c4afdaee0491ba04d19e2b31fd28f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:06:10.021799 32412 ts_tablet_manager.cc:1905] T dc50788aaa40455bb90570583ea51ff5 P 84410a7f07064d2eb11893a02950535f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:06:10.022621 32413 ts_tablet_manager.cc:1918] T dc50788aaa40455bb90570583ea51ff5 P ce9c4afdaee0491ba04d19e2b31fd28f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:06:10.022709 32413 log.cc:1196] T dc50788aaa40455bb90570583ea51ff5 P ce9c4afdaee0491ba04d19e2b31fd28f: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/wal/wals/dc50788aaa40455bb90570583ea51ff5
I20260812 03:06:10.023021 32413 ts_tablet_manager.cc:1939] T dc50788aaa40455bb90570583ea51ff5 P ce9c4afdaee0491ba04d19e2b31fd28f: Deleting consensus metadata
I20260812 03:06:10.024133 31613 catalog_manager.cc:4841] TS ce9c4afdaee0491ba04d19e2b31fd28f (127.5.96.131:36495): tablet dc50788aaa40455bb90570583ea51ff5 (table test-table [id=763c59dfd2a146eca8f39f3cf72bd25f]) successfully deleted
I20260812 03:06:10.024309 32412 ts_tablet_manager.cc:1918] T dc50788aaa40455bb90570583ea51ff5 P 84410a7f07064d2eb11893a02950535f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:06:10.024421 32412 log.cc:1196] T dc50788aaa40455bb90570583ea51ff5 P 84410a7f07064d2eb11893a02950535f: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/wal/wals/dc50788aaa40455bb90570583ea51ff5
I20260812 03:06:10.024749 32412 ts_tablet_manager.cc:1939] T dc50788aaa40455bb90570583ea51ff5 P 84410a7f07064d2eb11893a02950535f: Deleting consensus metadata
I20260812 03:06:10.025775 31611 catalog_manager.cc:4841] TS 84410a7f07064d2eb11893a02950535f (127.5.96.130:40785): tablet dc50788aaa40455bb90570583ea51ff5 (table test-table [id=763c59dfd2a146eca8f39f3cf72bd25f]) successfully deleted
W20260812 03:06:10.027374 31612 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.96.129:36745 (error 108)
I20260812 03:06:10.027590 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 31958
I20260812 03:06:10.042119 32378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:10.041809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:36745 (local address 127.0.0.1:44850)
0812 03:06:10.041938 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:10.042016 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.129:36745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 03:06:10.042294 31612 catalog_manager.cc:4584] TS cfcd528c8f4b40b5a86d56d912041895 (127.5.96.129:36745): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dc50788aaa40455bb90570583ea51ff5: Network error: Client connection negotiation failed: client connection to 127.5.96.129:36745: connect: Connection refused (error 111)
I20260812 03:06:10.046762 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 32154
I20260812 03:06:10.055653 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 31585
I20260812 03:06:30.067430 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:37671
--webserver_interface=127.5.96.190
--webserver_port=40421
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:40841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:37671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:06:30.174860 32416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:06:30.175081 32416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:06:30.175133 32416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:06:30.178963 32416 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 03:06:30.179005 32416 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 03:06:30.179026 32416 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 03:06:30.179044 32416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:06:30.179066 32416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:06:30.179095 32416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:06:30.179124 32416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:06:30.179147 32416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:06:30.184059 32416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:37671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:37671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=40421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:06:30.184934 32416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:06:30.185825 32416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:06:30.198674 32416 server_base.cc:1034] running on GCE node
W20260812 03:06:30.198673 32423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:06:30.198669 32425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:06:30.198844 32422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:06:30.199438 32416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:06:30.200482 32416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:06:30.201632 32416 hybrid_clock.cc:648] HybridClock initialized: now 1786503990201612 us; error 32 us; skew 500 ppm
Aug 12 03:06:30 dist-test-slave-6bbx krb5kdc[31557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503990, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:06:30.209224 32416 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:06:30.210057 32416 webserver.cc:460] Webserver started at http://127.5.96.190:40421/ using document root <none> and password file <none>
I20260812 03:06:30.210573 32416 fs_manager.cc:359] Metadata directory not provided
I20260812 03:06:30.210647 32416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:06:30.214314 32416 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260812 03:06:30.216307 32435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:06:30.217391 32416 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:06:30.217507 32416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "10658f3d49094703bf7640992e28bdcc"
format_stamp: "Formatted at 2026-08-12 03:06:09 on dist-test-slave-6bbx"
server_key: "a21426cbe8d38e227a147aa9ffdd3863"
server_key_iv: "c840d62ddd2f3b3a6e553caff045006b"
server_key_version: "encryptionkey@0"
I20260812 03:06:30.218077 32416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:06:30.243726 32416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:06:30.246176 32416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:06:30.246313 32416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:06:30.253263 32416 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:37671
I20260812 03:06:30.253518 32520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:37671 every 8 connection(s)
I20260812 03:06:30.254294 32416 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:06:30.258618 32522 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:06:30.260248 32522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc: Bootstrap starting.
I20260812 03:06:30.264238 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 32416
I20260812 03:06:30.264632 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:36745
--local_ip_for_outbound_sockets=127.5.96.129
--tserver_master_addrs=127.5.96.190:37671
--webserver_port=43481
--webserver_interface=127.5.96.129
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:40841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:06:30.277226 32522 log.cc:824] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc: Log is configured to *not* fsync() on all Append() calls
I20260812 03:06:30.285697 32522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:06:30.286019 32522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc: Bootstrap complete.
I20260812 03:06:30.289819 32522 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10658f3d49094703bf7640992e28bdcc" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37671 } }
I20260812 03:06:30.290194 32522 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10658f3d49094703bf7640992e28bdcc, State: Initialized, Role: FOLLOWER
I20260812 03:06:30.290678 32522 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10658f3d49094703bf7640992e28bdcc" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37671 } }
I20260812 03:06:30.290817 32522 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:06:30.290887 32522 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:06:30.290987 32522 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:06:30.292863 32522 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10658f3d49094703bf7640992e28bdcc" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37671 } }
I20260812 03:06:30.293174 32522 leader_election.cc:302] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10658f3d49094703bf7640992e28bdcc; no voters:
I20260812 03:06:30.293442 32522 leader_election.cc:288] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 03:06:30.293560 32526 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:06:30.293793 32526 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [term 2 LEADER]: Becoming Leader. State: Replica: 10658f3d49094703bf7640992e28bdcc, State: Running, Role: LEADER
I20260812 03:06:30.294160 32526 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10658f3d49094703bf7640992e28bdcc" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37671 } }
I20260812 03:06:30.294538 32522 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:06:30.295521 32527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "10658f3d49094703bf7640992e28bdcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10658f3d49094703bf7640992e28bdcc" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37671 } } }
I20260812 03:06:30.295641 32527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [sys.catalog]: This master's current role is: LEADER
I20260812 03:06:30.296015 32528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10658f3d49094703bf7640992e28bdcc. Latest consensus state: current_term: 2 leader_uuid: "10658f3d49094703bf7640992e28bdcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10658f3d49094703bf7640992e28bdcc" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37671 } } }
I20260812 03:06:30.296151 32528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [sys.catalog]: This master's current role is: LEADER
I20260812 03:06:30.296454 32544 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:06:30.301074 32544 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:06:30.301667 32544 catalog_manager.cc:1259] Loaded cluster ID: eaf0afac3404484fa868fcdba00f42be
I20260812 03:06:30.301729 32544 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:06:30.304126 32544 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:06:30.304797 32544 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc: Loaded TSK: 0
I20260812 03:06:30.335497 32544 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc: Generated new TSK 1
I20260812 03:06:30.335764 32544 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:06:30.383561 32524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:06:30.383826 32524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:06:30.383878 32524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:06:30.387763 32524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:06:30.387807 32524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:06:30.387869 32524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:06:30.392793 32524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:36745
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=43481
--tserver_master_addrs=127.5.96.190:37671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:06:30.393674 32524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:06:30.394593 32524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:06:30.403756 32570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:06:30.403761 32573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:06:30.403893 32575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:06:30.404230 32524 server_base.cc:1034] running on GCE node
I20260812 03:06:30.404583 32524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:06:30.405145 32524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:06:30.406311 32524 hybrid_clock.cc:648] HybridClock initialized: now 1786503990406263 us; error 65 us; skew 500 ppm
Aug 12 03:06:30 dist-test-slave-6bbx krb5kdc[31557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503990, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:06:30.408988 32524 init.cc:377] Logged in from keytab as kudu/127.5.96.129@KRBTEST.COM (short username kudu)
I20260812 03:06:30.409799 32524 webserver.cc:460] Webserver started at http://127.5.96.129:43481/ using document root <none> and password file <none>
I20260812 03:06:30.410310 32524 fs_manager.cc:359] Metadata directory not provided
I20260812 03:06:30.410386 32524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:06:30.414004 32524 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:06:30.416051 32585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:06:30.417244 32524 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:06:30.417362 32524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "cfcd528c8f4b40b5a86d56d912041895"
format_stamp: "Formatted at 2026-08-12 03:06:09 on dist-test-slave-6bbx"
server_key: "c3ae28dcaf27d913bc52b134a194cb81"
server_key_iv: "9636a49d4125b56badd25a674f9b3575"
server_key_version: "encryptionkey@0"
I20260812 03:06:30.417832 32524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:06:30.438900 32524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:06:30.441748 32524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:06:30.441911 32524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:06:30.442456 32524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:06:30.443423 32596 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 03:06:30.447366 32524 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 03:06:30.447422 32524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260812 03:06:30.447487 32524 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 03:06:30.449151 32524 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 03:06:30.449198 32524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260812 03:06:30.449481 32596 tablet_bootstrap.cc:492] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895: Bootstrap starting.
I20260812 03:06:30.457868 32524 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:36745
I20260812 03:06:30.457933 32763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:36745 every 8 connection(s)
I20260812 03:06:30.458736 32524 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:06:30.461397 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 32524
I20260812 03:06:30.461900 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:40785
--local_ip_for_outbound_sockets=127.5.96.130
--tserver_master_addrs=127.5.96.190:37671
--webserver_port=39199
--webserver_interface=127.5.96.130
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:40841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:06:30.468911 32596 log.cc:824] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895: Log is configured to *not* fsync() on all Append() calls
Aug 12 03:06:30 dist-test-slave-6bbx krb5kdc[31557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503990, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:06:30.476554 32596 tablet_bootstrap.cc:492] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:06:30.476985 32596 tablet_bootstrap.cc:492] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895: Bootstrap complete.
I20260812 03:06:30.478039 32596 ts_tablet_manager.cc:1397] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895: Time spent bootstrapping tablet: real 0.029s user 0.015s sys 0.012s
I20260812 03:06:30.478783 32766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:30.460871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.129:52339)
0812 03:06:30.463964 (+ 3093us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:30.464019 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 03:06:30.464950 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:06:30.466282 (+ 1332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:06:30.466289 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:06:30.466608 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:06:30.467053 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:30.467066 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.468382 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.468384 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:30.469339 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:30.469351 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.470347 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.470351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:30.470531 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:30.471172 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:06:30.471192 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:06:30.473042 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:06:30.475419 (+ 2377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:06:30.475425 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:06:30.475439 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:06:30.475821 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:06:30.476246 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:06:30.476249 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:06:30.476251 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:06:30.476416 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:06:30.477129 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:06:30.477135 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:06:30.477411 (+ 276us) client_negotiation.cc:769] Sending connection context
0812 03:06:30.477587 (+ 176us) client_negotiation.cc:240] Negotiation successful
0812 03:06:30.477884 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2866,"thread_start_us":149,"threads_started":1}
I20260812 03:06:30.478971 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:30.461035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:52339 (local address 127.5.96.190:37671)
0812 03:06:30.461542 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0812 03:06:30.461562 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 03:06:30.464161 (+ 2599us) server_negotiation.cc:407] Connection header received
0812 03:06:30.465447 (+ 1286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:06:30.465462 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:06:30.465870 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:06:30.466251 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:06:30.467212 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.468247 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.469474 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.470184 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.470376 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:30.473251 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:06:30.473258 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:06:30.473272 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:06:30.473303 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:06:30.475291 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:06:30.475958 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:06:30.475964 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:06:30.475969 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:06:30.476120 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:06:30.476521 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:06:30.476524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:06:30.476526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:06:30.476946 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:06:30.477060 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 03:06:30.477761 (+ 701us) server_negotiation.cc:299] Negotiation successful
0812 03:06:30.478051 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":98,"threads_started":1}
I20260812 03:06:30.480726 32596 raft_consensus.cc:348] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } }
I20260812 03:06:30.481194 32596 raft_consensus.cc:729] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfcd528c8f4b40b5a86d56d912041895, State: Initialized, Role: FOLLOWER
I20260812 03:06:30.481757 32596 consensus_queue.cc:260] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } }
I20260812 03:06:30.481767 32764 heartbeater.cc:347] Connected to a master server at 127.5.96.190:37671
I20260812 03:06:30.482302 32764 heartbeater.cc:464] Registering TS with master...
I20260812 03:06:30.482743 32596 ts_tablet_manager.cc:1428] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260812 03:06:30.482816 32764 heartbeater.cc:511] Master 127.5.96.190:37671 requested a full tablet report, sending...
I20260812 03:06:30.485215 32467 ts_manager.cc:194] Registered new tserver with Master: cfcd528c8f4b40b5a86d56d912041895 (127.5.96.129:36745)
I20260812 03:06:30.488415 32467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.129@KRBTEST.COM'} at 127.5.96.129:52339
I20260812 03:06:30.490702 32764 heartbeater.cc:503] Master 127.5.96.190:37671 was elected leader, sending a full tablet report...
Aug 12 03:06:30 dist-test-slave-6bbx krb5kdc[31557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503990, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for kudu/127.5.96.129@KRBTEST.COM
I20260812 03:06:30.497714 303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:30.487817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:36745 (local address 127.0.0.1:42018)
0812 03:06:30.488507 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:30.488552 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:06:30.488686 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:06:30.489187 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:06:30.489193 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:06:30.489569 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:06:30.489684 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:30.489692 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.491005 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.491008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:30.491702 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:30.491713 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.492308 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.492312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:30.492423 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:30.492884 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:06:30.492899 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:06:30.494372 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:06:30.496086 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:06:30.496096 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:06:30.496101 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:06:30.496395 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:06:30.496632 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:06:30.496635 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:06:30.496637 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:06:30.496731 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:06:30.497088 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:06:30.497094 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:06:30.497209 (+ 115us) client_negotiation.cc:769] Sending connection context
0812 03:06:30.497381 (+ 172us) client_negotiation.cc:240] Negotiation successful
0812 03:06:30.497555 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":98,"threads_started":1}
I20260812 03:06:30.497763 305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:30.487885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42018 (local address 127.5.96.129:36745)
0812 03:06:30.488261 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0812 03:06:30.488272 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:06:30.488638 (+ 366us) server_negotiation.cc:407] Connection header received
0812 03:06:30.488825 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:06:30.488830 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:06:30.489038 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:06:30.489177 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:06:30.489810 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.490883 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.491803 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.492195 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.492299 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:30.494500 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:06:30.494508 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:06:30.494516 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:06:30.494537 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:06:30.495980 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:06:30.496486 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:06:30.496489 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:06:30.496491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:06:30.496560 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:06:30.496805 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:06:30.496807 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:06:30.496809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:06:30.496997 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:06:30.497087 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 03:06:30.497460 (+ 373us) server_negotiation.cc:299] Negotiation successful
0812 03:06:30.497608 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":115,"threads_started":1}
I20260812 03:06:30.498801 32653 tablet_service.cc:1514] Processing DeleteTablet for tablet dc50788aaa40455bb90570583ea51ff5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:06:10 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:42018
I20260812 03:06:30.499517 306 tablet_replica.cc:330] stopping tablet replica
I20260812 03:06:30.499838 306 raft_consensus.cc:2229] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:06:30.500073 306 raft_consensus.cc:2258] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:06:30.501039 306 ts_tablet_manager.cc:1905] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:06:30.505406 306 ts_tablet_manager.cc:1918] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:06:30.505502 306 log.cc:1196] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/wal/wals/dc50788aaa40455bb90570583ea51ff5
I20260812 03:06:30.505764 306 ts_tablet_manager.cc:1939] T dc50788aaa40455bb90570583ea51ff5 P cfcd528c8f4b40b5a86d56d912041895: Deleting consensus metadata
I20260812 03:06:30.507445 32440 catalog_manager.cc:4841] TS cfcd528c8f4b40b5a86d56d912041895 (127.5.96.129:36745): tablet dc50788aaa40455bb90570583ea51ff5 (table test-table [id=763c59dfd2a146eca8f39f3cf72bd25f]) successfully deleted
W20260812 03:06:30.584273 300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:06:30.584520 300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:06:30.584563 300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:06:30.588513 300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:06:30.588567 300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:06:30.588635 300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:06:30.593770 300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:40785
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=39199
--tserver_master_addrs=127.5.96.190:37671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:06:30.594767 300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:06:30.595911 300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:06:30.605273 313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:06:30.605279 315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:06:30.605413 300 server_base.cc:1034] running on GCE node
W20260812 03:06:30.605623 311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:06:30.606045 300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:06:30.606631 300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:06:30.607785 300 hybrid_clock.cc:648] HybridClock initialized: now 1786503990607766 us; error 28 us; skew 500 ppm
Aug 12 03:06:30 dist-test-slave-6bbx krb5kdc[31557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503990, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:06:30.610311 300 init.cc:377] Logged in from keytab as kudu/127.5.96.130@KRBTEST.COM (short username kudu)
I20260812 03:06:30.611164 300 webserver.cc:460] Webserver started at http://127.5.96.130:39199/ using document root <none> and password file <none>
I20260812 03:06:30.611835 300 fs_manager.cc:359] Metadata directory not provided
I20260812 03:06:30.611917 300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:06:30.615582 300 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:06:30.617610 323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:06:30.618795 300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:06:30.618893 300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "84410a7f07064d2eb11893a02950535f"
format_stamp: "Formatted at 2026-08-12 03:06:09 on dist-test-slave-6bbx"
server_key: "64c6f023da0804f0b7c2192356efc641"
server_key_iv: "7546d210ba91b0581c39fbba40607246"
server_key_version: "encryptionkey@0"
I20260812 03:06:30.619427 300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:06:30.647261 300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:06:30.650038 300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:06:30.650453 300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:06:30.651041 300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:06:30.651886 300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:06:30.651933 300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:06:30.651973 300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:06:30.652002 300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:06:30.660377 300 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:40785
I20260812 03:06:30.660416 493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:40785 every 8 connection(s)
I20260812 03:06:30.661356 300 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
Aug 12 03:06:30 dist-test-slave-6bbx krb5kdc[31557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503990, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:06:30.671504 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 300
I20260812 03:06:30.672019 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:36495
--local_ip_for_outbound_sockets=127.5.96.131
--tserver_master_addrs=127.5.96.190:37671
--webserver_port=33305
--webserver_interface=127.5.96.131
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:40841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:06:30.674804 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:30.663082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:33085 (local address 127.5.96.190:37671)
0812 03:06:30.663198 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 03:06:30.663202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:06:30.664017 (+ 815us) server_negotiation.cc:407] Connection header received
0812 03:06:30.665469 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:06:30.665473 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:06:30.665557 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:06:30.665639 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:06:30.666962 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.667743 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.668869 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.669118 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.669208 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:30.671456 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:06:30.671460 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:06:30.671462 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:06:30.671489 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:06:30.672864 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:06:30.673338 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:06:30.673342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:06:30.673343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:06:30.673427 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:06:30.673731 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:06:30.673735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:06:30.673736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:06:30.673923 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:06:30.674010 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:06:30.674491 (+ 481us) server_negotiation.cc:299] Negotiation successful
0812 03:06:30.674625 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 03:06:30.675550 497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:30.663415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.130:33085)
0812 03:06:30.663877 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:30.663917 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:06:30.665201 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:06:30.665838 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:06:30.665847 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:06:30.666302 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:06:30.666782 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:30.666800 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.667881 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.667884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:30.668746 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:30.668757 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.669267 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.669271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:30.669435 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:30.669911 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:06:30.669926 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:06:30.671301 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:06:30.672970 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:06:30.672975 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:06:30.672989 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:06:30.673235 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:06:30.673509 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:06:30.673512 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:06:30.673514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:06:30.673645 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:06:30.674022 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:06:30.674028 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:06:30.674266 (+ 238us) client_negotiation.cc:769] Sending connection context
0812 03:06:30.674428 (+ 162us) client_negotiation.cc:240] Negotiation successful
0812 03:06:30.674738 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":128,"threads_started":1}
I20260812 03:06:30.676666 494 heartbeater.cc:347] Connected to a master server at 127.5.96.190:37671
I20260812 03:06:30.676931 494 heartbeater.cc:464] Registering TS with master...
I20260812 03:06:30.677508 494 heartbeater.cc:511] Master 127.5.96.190:37671 requested a full tablet report, sending...
I20260812 03:06:30.678682 32467 ts_manager.cc:194] Registered new tserver with Master: 84410a7f07064d2eb11893a02950535f (127.5.96.130:40785)
I20260812 03:06:30.679438 32467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.130@KRBTEST.COM'} at 127.5.96.130:33085
W20260812 03:06:30.784302 498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:06:30.784536 498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:06:30.784577 498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:06:30.788482 498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:06:30.788532 498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:06:30.788584 498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:06:30.793586 498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:36495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=33305
--tserver_master_addrs=127.5.96.190:37671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:06:30.794485 498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:06:30.795449 498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:06:30.804854 498 server_base.cc:1034] running on GCE node
W20260812 03:06:30.804837 505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:06:30.804922 508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:06:30.804831 504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:06:30.805447 498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:06:30.806063 498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:06:30.807269 498 hybrid_clock.cc:648] HybridClock initialized: now 1786503990807246 us; error 37 us; skew 500 ppm
Aug 12 03:06:30 dist-test-slave-6bbx krb5kdc[31557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503990, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:06:30.809873 498 init.cc:377] Logged in from keytab as kudu/127.5.96.131@KRBTEST.COM (short username kudu)
I20260812 03:06:30.810634 498 webserver.cc:460] Webserver started at http://127.5.96.131:33305/ using document root <none> and password file <none>
I20260812 03:06:30.811152 498 fs_manager.cc:359] Metadata directory not provided
I20260812 03:06:30.811242 498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:06:30.814860 498 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:06:30.816829 515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:06:30.817998 498 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:06:30.818111 498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "ce9c4afdaee0491ba04d19e2b31fd28f"
format_stamp: "Formatted at 2026-08-12 03:06:09 on dist-test-slave-6bbx"
server_key: "0cb1831e4bbf339c6ed44118c874d5ca"
server_key_iv: "50952497d09213753ac87783dec2c35a"
server_key_version: "encryptionkey@0"
I20260812 03:06:30.818615 498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:06:30.843709 498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:06:30.846345 498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:06:30.846514 498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:06:30.847082 498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:06:30.847954 498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:06:30.848004 498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:06:30.848058 498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:06:30.848083 498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:06:30.856618 498 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:36495
I20260812 03:06:30.856673 698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:36495 every 8 connection(s)
I20260812 03:06:30.857551 498 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:06:30.859175 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 498
Aug 12 03:06:30 dist-test-slave-6bbx krb5kdc[31557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786503990, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:06:30.868613 703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:30.859773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33140 (local address 127.5.96.190:37671)
0812 03:06:30.860018 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 03:06:30.860023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:06:30.860377 (+ 354us) server_negotiation.cc:407] Connection header received
0812 03:06:30.860418 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:06:30.860420 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:06:30.860462 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:06:30.860527 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:06:30.861489 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.862138 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.863296 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.863704 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.863806 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:30.865326 (+ 1520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:06:30.865329 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:06:30.865331 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:06:30.865356 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:06:30.866849 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:06:30.867346 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:06:30.867350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:06:30.867352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:06:30.867456 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:06:30.867716 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:06:30.867720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:06:30.867721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:06:30.867921 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:06:30.868069 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 03:06:30.868288 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 03:06:30.868438 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":180,"thread_start_us":86,"threads_started":1}
I20260812 03:06:30.871704 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:30.859215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:45659 (local address 127.5.96.190:37671)
0812 03:06:30.859370 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 03:06:30.859374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:06:30.860246 (+ 872us) server_negotiation.cc:407] Connection header received
0812 03:06:30.861816 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:06:30.861819 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:06:30.861867 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:06:30.861947 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:06:30.863290 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.864001 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.865539 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.865832 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.865943 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:30.868292 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:06:30.868296 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:06:30.868299 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:06:30.868319 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:06:30.869961 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:06:30.870560 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:06:30.870562 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:06:30.870563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:06:30.870637 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:06:30.870890 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:06:30.870892 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:06:30.870893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:06:30.871031 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:06:30.871095 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 03:06:30.871487 (+ 392us) server_negotiation.cc:299] Negotiation successful
0812 03:06:30.871569 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":62}
I20260812 03:06:30.872506 702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:30.859592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.131:45659)
0812 03:06:30.860092 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:30.860142 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 03:06:30.861627 (+ 1485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:06:30.862099 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:06:30.862107 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:06:30.862551 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:06:30.863139 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:30.863158 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.864151 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.864154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:30.865424 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:30.865434 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.865998 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.866002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:30.866146 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:30.866576 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:06:30.866590 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:06:30.868140 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:06:30.870066 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:06:30.870071 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:06:30.870084 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:06:30.870421 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:06:30.870711 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:06:30.870713 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:06:30.870714 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:06:30.870834 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:06:30.871098 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:06:30.871102 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:06:30.871316 (+ 214us) client_negotiation.cc:769] Sending connection context
0812 03:06:30.871457 (+ 141us) client_negotiation.cc:240] Negotiation successful
0812 03:06:30.871719 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":117,"threads_started":1}
I20260812 03:06:30.873515 699 heartbeater.cc:347] Connected to a master server at 127.5.96.190:37671
I20260812 03:06:30.873720 699 heartbeater.cc:464] Registering TS with master...
I20260812 03:06:30.874157 699 heartbeater.cc:511] Master 127.5.96.190:37671 requested a full tablet report, sending...
I20260812 03:06:30.875100 32467 ts_manager.cc:194] Registered new tserver with Master: ce9c4afdaee0491ba04d19e2b31fd28f (127.5.96.131:36495)
I20260812 03:06:30.875622 32467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.131@KRBTEST.COM'} at 127.5.96.131:45659
I20260812 03:06:30.881887 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:06:30.885349 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:30.882445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33146 (local address 127.5.96.190:37671)
0812 03:06:30.882567 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 03:06:30.882571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:06:30.882742 (+ 171us) server_negotiation.cc:407] Connection header received
0812 03:06:30.882874 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:06:30.882877 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:06:30.882978 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:06:30.883056 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:06:30.883232 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.883738 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.884493 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.884695 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.884763 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:30.884970 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:06:30.885084 (+ 114us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 03:06:30.885262 (+ 178us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33146: token expired
Metrics: {"server-negotiator.queue_time_us":47}
W20260812 03:06:30.885419 32767 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33146: token expired
W20260812 03:06:30.885825 32361 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.5.96.190:37671)
I20260812 03:06:30.891724 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:30.886132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33156 (local address 127.5.96.190:37671)
0812 03:06:30.886236 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 03:06:30.886239 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:06:30.886251 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:06:30.886281 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:06:30.886283 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:06:30.886313 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:06:30.886371 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:06:30.886872 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.887317 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.888048 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.888245 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.888316 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:30.889406 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:06:30.889408 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:06:30.889410 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:06:30.889424 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:06:30.890507 (+ 1083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:06:30.890853 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:06:30.890855 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:06:30.890857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:06:30.890926 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:06:30.891102 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:06:30.891104 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:06:30.891105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:06:30.891250 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:06:30.891342 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 03:06:30.891491 (+ 149us) server_negotiation.cc:299] Negotiation successful
0812 03:06:30.891573 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:06:30.893904 32467 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:06:30.897351 32467 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:06:30.905202 32653 tablet_service.cc:1467] Processing CreateTablet for tablet f48ad560e26446c99c7765d57c3de980 (DEFAULT_TABLE table=test-table [id=52bc26a8857a4c8aae155f1088e3acdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:06:30.905804 32653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f48ad560e26446c99c7765d57c3de980. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:06:30.908775 32596 tablet_bootstrap.cc:492] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895: Bootstrap starting.
I20260812 03:06:30.909564 32596 tablet_bootstrap.cc:654] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895: Neither blocks nor log segments found. Creating new log.
I20260812 03:06:30.911145 32596 tablet_bootstrap.cc:492] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895: No bootstrap required, opened a new log
I20260812 03:06:30.911239 32596 ts_tablet_manager.cc:1397] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260812 03:06:30.911276 303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:30.903754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:40785 (local address 127.0.0.1:49454)
0812 03:06:30.903971 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:30.903987 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:06:30.904077 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:06:30.907242 (+ 3165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:06:30.907246 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:06:30.907266 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:06:30.907386 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:30.907393 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.909013 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.909017 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:30.909891 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:30.909901 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.910839 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.910843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:30.910940 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:30.910994 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 03:06:30.911049 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 03:06:30.911131 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":17}
I20260812 03:06:30.911731 32596 raft_consensus.cc:348] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } }
I20260812 03:06:30.911865 32596 raft_consensus.cc:374] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:06:30.911904 32596 raft_consensus.cc:729] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfcd528c8f4b40b5a86d56d912041895, State: Initialized, Role: FOLLOWER
I20260812 03:06:30.912034 32596 consensus_queue.cc:260] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } }
I20260812 03:06:30.912209 710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:30.903946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49454 (local address 127.5.96.130:40785)
0812 03:06:30.906673 (+ 2727us) server_negotiation.cc:206] Beginning negotiation
0812 03:06:30.906686 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:06:30.906712 (+ 26us) server_negotiation.cc:407] Connection header received
0812 03:06:30.906798 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:06:30.906803 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:06:30.907059 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:06:30.907550 (+ 491us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:06:30.907745 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.908788 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.910007 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.910701 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.911527 (+ 826us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:30.911708 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 03:06:30.911767 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 03:06:30.911863 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2610,"thread_start_us":2047,"threads_started":1}
I20260812 03:06:30.912302 711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:30.904722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:36495 (local address 127.0.0.1:34730)
0812 03:06:30.906836 (+ 2114us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:30.906849 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:06:30.906946 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:06:30.907525 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:06:30.907528 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:06:30.907548 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:06:30.907636 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:30.907642 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.909015 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.909019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:30.910745 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:30.910755 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.911887 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.911890 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:30.911993 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:30.912016 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:06:30.912080 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 03:06:30.912165 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2056,"mutex_wait_us":3,"thread_start_us":1897,"threads_started":1}
I20260812 03:06:30.912362 712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:30.906294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34730 (local address 127.5.96.131:36495)
0812 03:06:30.906711 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0812 03:06:30.906721 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:06:30.906935 (+ 214us) server_negotiation.cc:407] Connection header received
0812 03:06:30.907054 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:06:30.907060 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:06:30.907255 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:06:30.907433 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:06:30.907824 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.908893 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.910881 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.911579 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.911643 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:30.911795 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 03:06:30.912140 (+ 345us) server_negotiation.cc:299] Negotiation successful
0812 03:06:30.912232 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":158,"threads_started":1}
I20260812 03:06:30.913344 32596 ts_tablet_manager.cc:1428] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:06:30.915375 597 tablet_service.cc:1467] Processing CreateTablet for tablet f48ad560e26446c99c7765d57c3de980 (DEFAULT_TABLE table=test-table [id=52bc26a8857a4c8aae155f1088e3acdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:06:30.915339 393 tablet_service.cc:1467] Processing CreateTablet for tablet f48ad560e26446c99c7765d57c3de980 (DEFAULT_TABLE table=test-table [id=52bc26a8857a4c8aae155f1088e3acdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:06:30.916316 393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f48ad560e26446c99c7765d57c3de980. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:06:30.916325 597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f48ad560e26446c99c7765d57c3de980. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:06:30.922991 714 tablet_bootstrap.cc:492] T f48ad560e26446c99c7765d57c3de980 P ce9c4afdaee0491ba04d19e2b31fd28f: Bootstrap starting.
I20260812 03:06:30.923853 715 tablet_bootstrap.cc:492] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f: Bootstrap starting.
I20260812 03:06:30.925590 714 tablet_bootstrap.cc:654] T f48ad560e26446c99c7765d57c3de980 P ce9c4afdaee0491ba04d19e2b31fd28f: Neither blocks nor log segments found. Creating new log.
I20260812 03:06:30.926345 715 tablet_bootstrap.cc:654] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f: Neither blocks nor log segments found. Creating new log.
I20260812 03:06:30.926971 714 log.cc:824] T f48ad560e26446c99c7765d57c3de980 P ce9c4afdaee0491ba04d19e2b31fd28f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:06:30.927089 715 log.cc:824] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:06:30.929088 715 tablet_bootstrap.cc:492] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f: No bootstrap required, opened a new log
I20260812 03:06:30.929100 714 tablet_bootstrap.cc:492] T f48ad560e26446c99c7765d57c3de980 P ce9c4afdaee0491ba04d19e2b31fd28f: No bootstrap required, opened a new log
I20260812 03:06:30.929265 714 ts_tablet_manager.cc:1397] T f48ad560e26446c99c7765d57c3de980 P ce9c4afdaee0491ba04d19e2b31fd28f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:06:30.929286 715 ts_tablet_manager.cc:1397] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:06:30.932049 715 raft_consensus.cc:348] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } }
I20260812 03:06:30.932022 714 raft_consensus.cc:348] T f48ad560e26446c99c7765d57c3de980 P ce9c4afdaee0491ba04d19e2b31fd28f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } }
I20260812 03:06:30.932287 714 raft_consensus.cc:374] T f48ad560e26446c99c7765d57c3de980 P ce9c4afdaee0491ba04d19e2b31fd28f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:06:30.932291 715 raft_consensus.cc:374] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:06:30.932345 715 raft_consensus.cc:729] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84410a7f07064d2eb11893a02950535f, State: Initialized, Role: FOLLOWER
I20260812 03:06:30.932344 714 raft_consensus.cc:729] T f48ad560e26446c99c7765d57c3de980 P ce9c4afdaee0491ba04d19e2b31fd28f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce9c4afdaee0491ba04d19e2b31fd28f, State: Initialized, Role: FOLLOWER
I20260812 03:06:30.932763 715 consensus_queue.cc:260] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } }
I20260812 03:06:30.932787 714 consensus_queue.cc:260] T f48ad560e26446c99c7765d57c3de980 P ce9c4afdaee0491ba04d19e2b31fd28f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } }
I20260812 03:06:30.933468 494 heartbeater.cc:503] Master 127.5.96.190:37671 was elected leader, sending a full tablet report...
I20260812 03:06:30.933672 715 ts_tablet_manager.cc:1428] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260812 03:06:30.933678 714 ts_tablet_manager.cc:1428] T f48ad560e26446c99c7765d57c3de980 P ce9c4afdaee0491ba04d19e2b31fd28f: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260812 03:06:30.933997 699 heartbeater.cc:503] Master 127.5.96.190:37671 was elected leader, sending a full tablet report...
W20260812 03:06:30.959884 32765 tablet.cc:2369] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:06:30.982111 719 raft_consensus.cc:482] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:06:30.982371 719 raft_consensus.cc:504] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } }
I20260812 03:06:30.983556 719 leader_election.cc:288] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce9c4afdaee0491ba04d19e2b31fd28f (127.5.96.131:36495), cfcd528c8f4b40b5a86d56d912041895 (127.5.96.129:36745)
I20260812 03:06:30.988173 305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:30.983965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:53463 (local address 127.5.96.129:36745)
0812 03:06:30.984093 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 03:06:30.984098 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:06:30.984115 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:06:30.984173 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:06:30.984176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:06:30.984243 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:06:30.984350 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:06:30.984682 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.985458 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.986977 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.987704 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.987811 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:30.987960 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 03:06:30.987993 (+ 33us) server_negotiation.cc:299] Negotiation successful
0812 03:06:30.988061 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:06:30.988190 712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:30.983973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:60777 (local address 127.5.96.131:36495)
0812 03:06:30.984104 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 03:06:30.984109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:06:30.984439 (+ 330us) server_negotiation.cc:407] Connection header received
0812 03:06:30.984538 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:06:30.984540 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:06:30.984580 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:06:30.984649 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:06:30.984879 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.985592 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.986974 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.987578 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.987729 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:30.987788 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:06:30.987961 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 03:06:30.988038 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 03:06:30.988175 497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:30.983869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:36745 (local address 127.5.96.130:53463)
0812 03:06:30.983984 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:30.983999 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:06:30.984086 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:06:30.984385 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:06:30.984388 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:06:30.984410 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:06:30.984557 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:30.984564 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.985569 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.985571 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:30.986839 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:30.986851 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.987803 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.987806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:30.987895 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:30.987912 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 03:06:30.987963 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 03:06:30.988011 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":1,"spinlock_wait_cycles":16128}
I20260812 03:06:30.988183 722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:30.983878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:36495 (local address 127.5.96.130:60777)
0812 03:06:30.984341 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:30.984354 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:06:30.984462 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:06:30.984671 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:06:30.984674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:06:30.984691 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:06:30.984789 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:30.984793 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.985677 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.985679 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:30.986851 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:30.986858 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:30.987719 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:30.987721 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:30.987831 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:30.987859 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 03:06:30.987936 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 03:06:30.988008 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"mutex_wait_us":140,"spinlock_wait_cycles":1664,"thread_start_us":139,"threads_started":1}
I20260812 03:06:30.988837 32688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f48ad560e26446c99c7765d57c3de980" candidate_uuid: "84410a7f07064d2eb11893a02950535f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfcd528c8f4b40b5a86d56d912041895" is_pre_election: true
I20260812 03:06:30.988924 626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f48ad560e26446c99c7765d57c3de980" candidate_uuid: "84410a7f07064d2eb11893a02950535f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" is_pre_election: true
I20260812 03:06:30.989137 32688 raft_consensus.cc:2454] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84410a7f07064d2eb11893a02950535f in term 0.
I20260812 03:06:30.989195 626 raft_consensus.cc:2454] T f48ad560e26446c99c7765d57c3de980 P ce9c4afdaee0491ba04d19e2b31fd28f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84410a7f07064d2eb11893a02950535f in term 0.
I20260812 03:06:30.989646 327 leader_election.cc:302] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84410a7f07064d2eb11893a02950535f, cfcd528c8f4b40b5a86d56d912041895; no voters:
I20260812 03:06:30.989897 719 raft_consensus.cc:2781] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:06:30.989990 719 raft_consensus.cc:482] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:06:30.990039 719 raft_consensus.cc:3037] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:06:30.991047 719 raft_consensus.cc:504] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } }
I20260812 03:06:30.991369 719 leader_election.cc:288] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f [CANDIDATE]: Term 1 election: Requested vote from peers ce9c4afdaee0491ba04d19e2b31fd28f (127.5.96.131:36495), cfcd528c8f4b40b5a86d56d912041895 (127.5.96.129:36745)
I20260812 03:06:30.991707 32688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f48ad560e26446c99c7765d57c3de980" candidate_uuid: "84410a7f07064d2eb11893a02950535f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfcd528c8f4b40b5a86d56d912041895"
I20260812 03:06:30.991727 626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f48ad560e26446c99c7765d57c3de980" candidate_uuid: "84410a7f07064d2eb11893a02950535f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f"
I20260812 03:06:30.991832 626 raft_consensus.cc:3037] T f48ad560e26446c99c7765d57c3de980 P ce9c4afdaee0491ba04d19e2b31fd28f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:06:30.991837 32688 raft_consensus.cc:3037] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:06:30.992798 32688 raft_consensus.cc:2454] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84410a7f07064d2eb11893a02950535f in term 1.
I20260812 03:06:30.992869 626 raft_consensus.cc:2454] T f48ad560e26446c99c7765d57c3de980 P ce9c4afdaee0491ba04d19e2b31fd28f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84410a7f07064d2eb11893a02950535f in term 1.
I20260812 03:06:30.993050 327 leader_election.cc:302] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84410a7f07064d2eb11893a02950535f, cfcd528c8f4b40b5a86d56d912041895; no voters:
I20260812 03:06:30.993214 719 raft_consensus.cc:2781] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:06:30.993427 719 raft_consensus.cc:686] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f [term 1 LEADER]: Becoming Leader. State: Replica: 84410a7f07064d2eb11893a02950535f, State: Running, Role: LEADER
I20260812 03:06:30.993736 719 consensus_queue.cc:237] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } }
I20260812 03:06:30.997009 32465 catalog_manager.cc:5495] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f reported cstate change: term changed from 0 to 1, leader changed from <none> to 84410a7f07064d2eb11893a02950535f (127.5.96.130). New cstate: current_term: 1 leader_uuid: "84410a7f07064d2eb11893a02950535f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } health_report { overall_health: UNKNOWN } } }
I20260812 03:06:31.008795 710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:31.005644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49470 (local address 127.5.96.130:40785)
0812 03:06:31.005766 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 03:06:31.005769 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:06:31.005782 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:06:31.005924 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:06:31.005926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:06:31.005973 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:06:31.006057 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:06:31.006237 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:31.006768 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:31.007605 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:31.007853 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:31.007925 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:31.008145 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:06:31.008526 (+ 381us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:06:31.008610 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 03:06:31.008636 (+ 26us) server_negotiation.cc:299] Negotiation successful
0812 03:06:31.008681 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:06:31.014456 626 raft_consensus.cc:1261] T f48ad560e26446c99c7765d57c3de980 P ce9c4afdaee0491ba04d19e2b31fd28f [term 1 FOLLOWER]: Refusing update from remote peer 84410a7f07064d2eb11893a02950535f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:06:31.014452 32688 raft_consensus.cc:1261] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [term 1 FOLLOWER]: Refusing update from remote peer 84410a7f07064d2eb11893a02950535f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:06:31.015161 719 consensus_queue.cc:1035] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:06:31.015368 723 consensus_queue.cc:1035] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:06:31.020213 726 mvcc.cc:205] Tried to move back new op lower bound from 7317520347187949568 to 7317520347112931328. Current Snapshot: MvccSnapshot[applied={T|T < 7317520347187949568}]
I20260812 03:06:31.020700 729 mvcc.cc:205] Tried to move back new op lower bound from 7317520347187949568 to 7317520347112931328. Current Snapshot: MvccSnapshot[applied={T|T < 7317520347187949568}]
I20260812 03:06:31.028784 32467 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:06:31.028926 32467 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:06:31.030980 32467 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc: Sending DeleteTablet for 3 replicas of tablet f48ad560e26446c99c7765d57c3de980
I20260812 03:06:31.031530 32653 tablet_service.cc:1514] Processing DeleteTablet for tablet f48ad560e26446c99c7765d57c3de980 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:06:31 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:42018
I20260812 03:06:31.031590 393 tablet_service.cc:1514] Processing DeleteTablet for tablet f48ad560e26446c99c7765d57c3de980 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:06:31 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:49454
I20260812 03:06:31.031829 597 tablet_service.cc:1514] Processing DeleteTablet for tablet f48ad560e26446c99c7765d57c3de980 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:06:31 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:34730
I20260812 03:06:31.032001 737 tablet_replica.cc:330] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895: stopping tablet replica
I20260812 03:06:31.032117 737 raft_consensus.cc:2229] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:06:31.032191 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 32524
I20260812 03:06:31.032234 737 raft_consensus.cc:2258] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:06:31.048003 738 tablet_replica.cc:330] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f: stopping tablet replica
I20260812 03:06:31.048409 739 tablet_replica.cc:330] T f48ad560e26446c99c7765d57c3de980 P ce9c4afdaee0491ba04d19e2b31fd28f: stopping tablet replica
I20260812 03:06:31.048426 738 raft_consensus.cc:2229] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:06:31.048839 739 raft_consensus.cc:2229] T f48ad560e26446c99c7765d57c3de980 P ce9c4afdaee0491ba04d19e2b31fd28f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:06:31.048879 738 raft_consensus.cc:2258] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:06:31.049639 739 raft_consensus.cc:2258] T f48ad560e26446c99c7765d57c3de980 P ce9c4afdaee0491ba04d19e2b31fd28f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:06:31.050516 738 ts_tablet_manager.cc:1905] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:06:31.051934 739 ts_tablet_manager.cc:1905] T f48ad560e26446c99c7765d57c3de980 P ce9c4afdaee0491ba04d19e2b31fd28f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 03:06:31.051944 32440 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.96.129:36745 (error 108)
I20260812 03:06:31.053181 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 300
I20260812 03:06:31.053372 738 ts_tablet_manager.cc:1918] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:06:31.056954 739 ts_tablet_manager.cc:1918] T f48ad560e26446c99c7765d57c3de980 P ce9c4afdaee0491ba04d19e2b31fd28f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:06:31.057050 739 log.cc:1196] T f48ad560e26446c99c7765d57c3de980 P ce9c4afdaee0491ba04d19e2b31fd28f: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/wal/wals/f48ad560e26446c99c7765d57c3de980
I20260812 03:06:31.057106 711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:31.056758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:36745 (local address 127.0.0.1:42024)
0812 03:06:31.056921 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:31.057009 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.129:36745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 03:06:31.057402 739 ts_tablet_manager.cc:1939] T f48ad560e26446c99c7765d57c3de980 P ce9c4afdaee0491ba04d19e2b31fd28f: Deleting consensus metadata
W20260812 03:06:31.057296 32440 catalog_manager.cc:4584] TS cfcd528c8f4b40b5a86d56d912041895 (127.5.96.129:36745): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f48ad560e26446c99c7765d57c3de980: Network error: Client connection negotiation failed: client connection to 127.5.96.129:36745: connect: Connection refused (error 111)
I20260812 03:06:31.058621 32441 catalog_manager.cc:4841] TS ce9c4afdaee0491ba04d19e2b31fd28f (127.5.96.131:36495): tablet f48ad560e26446c99c7765d57c3de980 (table test-table [id=52bc26a8857a4c8aae155f1088e3acdf]) successfully deleted
I20260812 03:06:31.062180 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 498
I20260812 03:06:31.062575 711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:31.062305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:40785 (local address 127.0.0.1:49480)
0812 03:06:31.062461 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:31.062505 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.130:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260812 03:06:31.072685 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 32416
I20260812 03:06:51.088115 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:37671
--webserver_interface=127.5.96.190
--webserver_port=40421
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:40841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:37671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:06:51.196208 741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:06:51.196420 741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:06:51.196456 741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:06:51.200249 741 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 03:06:51.200290 741 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 03:06:51.200304 741 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 03:06:51.200316 741 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:06:51.200335 741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:06:51.200356 741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:06:51.200371 741 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:06:51.200384 741 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:06:51.205279 741 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:37671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:37671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=40421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:06:51.206141 741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:06:51.207101 741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:06:51.214915 749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:06:51.214959 741 server_base.cc:1034] running on GCE node
W20260812 03:06:51.214931 748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:06:51.214922 753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:06:51.215637 741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:06:51.216619 741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:06:51.217775 741 hybrid_clock.cc:648] HybridClock initialized: now 1786504011217760 us; error 28 us; skew 500 ppm
Aug 12 03:06:51 dist-test-slave-6bbx krb5kdc[31557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504011, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:06:51.223112 741 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:06:51.223966 741 webserver.cc:460] Webserver started at http://127.5.96.190:40421/ using document root <none> and password file <none>
I20260812 03:06:51.224462 741 fs_manager.cc:359] Metadata directory not provided
I20260812 03:06:51.224527 741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:06:51.228017 741 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:06:51.229902 767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:06:51.231012 741 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:06:51.231122 741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "10658f3d49094703bf7640992e28bdcc"
format_stamp: "Formatted at 2026-08-12 03:06:09 on dist-test-slave-6bbx"
server_key: "a21426cbe8d38e227a147aa9ffdd3863"
server_key_iv: "c840d62ddd2f3b3a6e553caff045006b"
server_key_version: "encryptionkey@0"
I20260812 03:06:51.231590 741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:06:51.240244 741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:06:51.243038 741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:06:51.243196 741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:06:51.249739 741 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:37671
I20260812 03:06:51.249979 841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:37671 every 8 connection(s)
I20260812 03:06:51.250684 741 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:06:51.254619 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 741
I20260812 03:06:51.254720 842 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:06:51.255112 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:36745
--local_ip_for_outbound_sockets=127.5.96.129
--tserver_master_addrs=127.5.96.190:37671
--webserver_port=43481
--webserver_interface=127.5.96.129
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:40841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:06:51.256723 842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc: Bootstrap starting.
I20260812 03:06:51.273909 842 log.cc:824] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc: Log is configured to *not* fsync() on all Append() calls
I20260812 03:06:51.284837 842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:06:51.285166 842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc: Bootstrap complete.
I20260812 03:06:51.288954 842 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10658f3d49094703bf7640992e28bdcc" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37671 } }
I20260812 03:06:51.289314 842 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10658f3d49094703bf7640992e28bdcc, State: Initialized, Role: FOLLOWER
I20260812 03:06:51.289759 842 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10658f3d49094703bf7640992e28bdcc" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37671 } }
I20260812 03:06:51.289878 842 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:06:51.289942 842 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:06:51.290072 842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [term 2 FOLLOWER]: Advancing to term 3
I20260812 03:06:51.291857 842 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10658f3d49094703bf7640992e28bdcc" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37671 } }
I20260812 03:06:51.292156 842 leader_election.cc:302] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10658f3d49094703bf7640992e28bdcc; no voters:
I20260812 03:06:51.292428 842 leader_election.cc:288] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 03:06:51.292541 847 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [term 3 FOLLOWER]: Leader election won for term 3
I20260812 03:06:51.292755 847 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [term 3 LEADER]: Becoming Leader. State: Replica: 10658f3d49094703bf7640992e28bdcc, State: Running, Role: LEADER
I20260812 03:06:51.293114 847 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10658f3d49094703bf7640992e28bdcc" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37671 } }
I20260812 03:06:51.293550 842 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:06:51.294534 849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10658f3d49094703bf7640992e28bdcc. Latest consensus state: current_term: 3 leader_uuid: "10658f3d49094703bf7640992e28bdcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10658f3d49094703bf7640992e28bdcc" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37671 } } }
I20260812 03:06:51.294659 849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [sys.catalog]: This master's current role is: LEADER
I20260812 03:06:51.294981 848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "10658f3d49094703bf7640992e28bdcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10658f3d49094703bf7640992e28bdcc" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37671 } } }
I20260812 03:06:51.295043 857 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:06:51.295107 848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc [sys.catalog]: This master's current role is: LEADER
I20260812 03:06:51.300109 857 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:06:51.300792 857 catalog_manager.cc:1259] Loaded cluster ID: eaf0afac3404484fa868fcdba00f42be
I20260812 03:06:51.300851 857 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:06:51.303282 857 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:06:51.303994 857 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc: Loaded TSK: 0
I20260812 03:06:51.304076 857 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 10658f3d49094703bf7640992e28bdcc: Loaded TSK: 1
F20260812 03:06:51.304761 857 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f4f5b8659ad google::LogMessage::Fail() at ??:0
@ 0x7f4f5b86888a google::LogMessage::SendToLog() at ??:0
@ 0x7f4f5b8654de google::LogMessage::Flush() at ??:0
@ 0x7f4f5b866a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f4f5e52918d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f4f5e521f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f4f661ca7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f4f661a499c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f4f661a5178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f4f661e6df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f4f6620582b std::function<>::operator()() at ??:0
@ 0x7f4f661a4b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f4f661a5f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f4f661a2579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f4f661e6640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f4f5dab73d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f4f5dab7c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f4f5dab93f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f4f5daa8512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f4f5f0ec6db start_thread at ??:0
@ 0x7f4f5a36361f clone at ??:0
W20260812 03:06:51.387789 845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:06:51.387995 845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:06:51.388033 845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:06:51.392114 845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:06:51.392153 845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:06:51.392205 845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:06:51.397360 845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:36745
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=43481
--tserver_master_addrs=127.5.96.190:37671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:06:51.398433 845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:06:51.399375 845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:06:51.408704 912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:06:51.408757 913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:06:51.408964 916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:06:51.409026 845 server_base.cc:1034] running on GCE node
I20260812 03:06:51.409421 845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:06:51.410033 845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:06:51.411211 845 hybrid_clock.cc:648] HybridClock initialized: now 1786504011411188 us; error 32 us; skew 500 ppm
Aug 12 03:06:51 dist-test-slave-6bbx krb5kdc[31557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504011, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:06:51.413946 845 init.cc:377] Logged in from keytab as kudu/127.5.96.129@KRBTEST.COM (short username kudu)
I20260812 03:06:51.414728 845 webserver.cc:460] Webserver started at http://127.5.96.129:43481/ using document root <none> and password file <none>
I20260812 03:06:51.415234 845 fs_manager.cc:359] Metadata directory not provided
I20260812 03:06:51.415300 845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:06:51.418886 845 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:06:51.420864 929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:06:51.422043 845 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:06:51.422145 845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "cfcd528c8f4b40b5a86d56d912041895"
format_stamp: "Formatted at 2026-08-12 03:06:09 on dist-test-slave-6bbx"
server_key: "c3ae28dcaf27d913bc52b134a194cb81"
server_key_iv: "9636a49d4125b56badd25a674f9b3575"
server_key_version: "encryptionkey@0"
I20260812 03:06:51.422636 845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:06:51.446552 845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:06:51.449378 845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:06:51.449537 845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:06:51.450074 845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:06:51.451108 945 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 03:06:51.456355 845 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 03:06:51.456415 845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260812 03:06:51.456475 845 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 03:06:51.458038 845 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 03:06:51.458081 845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260812 03:06:51.458416 945 tablet_bootstrap.cc:492] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895: Bootstrap starting.
I20260812 03:06:51.467705 845 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:36745
I20260812 03:06:51.467842 1131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:36745 every 8 connection(s)
I20260812 03:06:51.468637 845 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:06:51.472262 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 845
I20260812 03:06:51.472653 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:40785
--local_ip_for_outbound_sockets=127.5.96.130
--tserver_master_addrs=127.5.96.190:37671
--webserver_port=39199
--webserver_interface=127.5.96.130
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:40841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:06:51.479733 945 log.cc:824] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895: Log is configured to *not* fsync() on all Append() calls
I20260812 03:06:51.488577 945 tablet_bootstrap.cc:492] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:06:51.488982 945 tablet_bootstrap.cc:492] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895: Bootstrap complete.
I20260812 03:06:51.490285 945 ts_tablet_manager.cc:1397] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895: Time spent bootstrapping tablet: real 0.032s user 0.022s sys 0.007s
I20260812 03:06:51.493383 945 raft_consensus.cc:348] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } }
I20260812 03:06:51.493923 945 raft_consensus.cc:729] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfcd528c8f4b40b5a86d56d912041895, State: Initialized, Role: FOLLOWER
I20260812 03:06:51.494485 945 consensus_queue.cc:260] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } }
I20260812 03:06:51.495522 945 ts_tablet_manager.cc:1428] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895: Time spent starting tablet: real 0.005s user 0.006s sys 0.001s
W20260812 03:06:51.538165 1135 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:06:51.470377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.129:35453)
0812 03:06:51.470830 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:51.470881 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 03:06:51.471647 (+ 766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:06:51.537433 (+ 65786us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":118,"threads_started":1}
W20260812 03:06:51.538648 1133 heartbeater.cc:644] Failed to heartbeat to 127.5.96.190:37671 (0 consecutive failures): Network error: Failed to ping master at 127.5.96.190:37671: Client connection negotiation failed: client connection to 127.5.96.190:37671: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 03:06:51.539400 1135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:51.539157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.129:47271)
0812 03:06:51.539282 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:51.539344 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 03:06:51.540035 1135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:51.539850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.129:57125)
0812 03:06:51.539944 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:51.539985 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 03:06:51.540235 1133 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:06:51.590180 1136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:06:51.590392 1136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:06:51.590431 1136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:06:51.594250 1136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:06:51.594295 1136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:06:51.594347 1136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:06:51.599206 1136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:40785
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=39199
--tserver_master_addrs=127.5.96.190:37671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:06:51.600044 1136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:06:51.600891 1136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:06:51.609688 1146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:06:51.609704 1148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:06:51.609692 1145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:06:51.609746 1136 server_base.cc:1034] running on GCE node
I20260812 03:06:51.610255 1136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:06:51.610797 1136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:06:51.611943 1136 hybrid_clock.cc:648] HybridClock initialized: now 1786504011611919 us; error 33 us; skew 500 ppm
Aug 12 03:06:51 dist-test-slave-6bbx krb5kdc[31557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504011, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:06:51.614413 1136 init.cc:377] Logged in from keytab as kudu/127.5.96.130@KRBTEST.COM (short username kudu)
I20260812 03:06:51.615214 1136 webserver.cc:460] Webserver started at http://127.5.96.130:39199/ using document root <none> and password file <none>
I20260812 03:06:51.615753 1136 fs_manager.cc:359] Metadata directory not provided
I20260812 03:06:51.615825 1136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:06:51.619256 1136 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:06:51.621240 1160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:06:51.622352 1136 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:06:51.622458 1136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "84410a7f07064d2eb11893a02950535f"
format_stamp: "Formatted at 2026-08-12 03:06:09 on dist-test-slave-6bbx"
server_key: "64c6f023da0804f0b7c2192356efc641"
server_key_iv: "7546d210ba91b0581c39fbba40607246"
server_key_version: "encryptionkey@0"
I20260812 03:06:51.622982 1136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:06:51.632337 1136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:06:51.634972 1136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:06:51.635140 1136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:06:51.635653 1136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:06:51.636660 1170 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260812 03:06:51.640122 1170 ts_tablet_manager.cc:1852] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260812 03:06:51.640486 1170 ts_tablet_manager.cc:1905] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:06:51.644014 1170 ts_tablet_manager.cc:1918] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:06:51.644106 1170 log.cc:1196] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/wal/wals/f48ad560e26446c99c7765d57c3de980
I20260812 03:06:51.644418 1170 ts_tablet_manager.cc:1939] T f48ad560e26446c99c7765d57c3de980 P 84410a7f07064d2eb11893a02950535f: Deleting consensus metadata
I20260812 03:06:51.644696 1136 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260812 03:06:51.644749 1136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.008s user 0.000s sys 0.000s
I20260812 03:06:51.644811 1136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:06:51.644836 1136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:06:51.652918 1136 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:40785
I20260812 03:06:51.652987 1338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:40785 every 8 connection(s)
I20260812 03:06:51.653709 1136 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:06:51.656656 1343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:51.655512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.130:47727)
0812 03:06:51.655942 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:51.656040 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":134,"threads_started":1}
W20260812 03:06:51.657073 1339 heartbeater.cc:644] Failed to heartbeat to 127.5.96.190:37671 (0 consecutive failures): Network error: Failed to ping master at 127.5.96.190:37671: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
I20260812 03:06:51.657560 1343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:51.657368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.130:58983)
0812 03:06:51.657465 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:51.657509 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 03:06:51.658116 1343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:51.657936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.130:55445)
0812 03:06:51.658030 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:51.658072 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 03:06:51.658260 1339 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 03:06:51.663837 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 1136
I20260812 03:06:51.664223 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:36495
--local_ip_for_outbound_sockets=127.5.96.131
--tserver_master_addrs=127.5.96.190:37671
--webserver_port=33305
--webserver_interface=127.5.96.131
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:40841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:06:51.771188 1344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:06:51.771399 1344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:06:51.771437 1344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:06:51.775327 1344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:06:51.775378 1344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:06:51.775435 1344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:06:51.780330 1344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:36495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=33305
--tserver_master_addrs=127.5.96.190:37671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:06:51.781172 1344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:06:51.782063 1344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:06:51.790805 1355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:06:51.790815 1344 server_base.cc:1034] running on GCE node
W20260812 03:06:51.790805 1352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:06:51.790812 1353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:06:51.791359 1344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:06:51.791919 1344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:06:51.793087 1344 hybrid_clock.cc:648] HybridClock initialized: now 1786504011793063 us; error 34 us; skew 500 ppm
Aug 12 03:06:51 dist-test-slave-6bbx krb5kdc[31557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504011, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:06:51.795495 1344 init.cc:377] Logged in from keytab as kudu/127.5.96.131@KRBTEST.COM (short username kudu)
I20260812 03:06:51.796315 1344 webserver.cc:460] Webserver started at http://127.5.96.131:33305/ using document root <none> and password file <none>
I20260812 03:06:51.796818 1344 fs_manager.cc:359] Metadata directory not provided
I20260812 03:06:51.796882 1344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:06:51.800299 1344 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:06:51.802248 1363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:06:51.803310 1344 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:06:51.803413 1344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "ce9c4afdaee0491ba04d19e2b31fd28f"
format_stamp: "Formatted at 2026-08-12 03:06:09 on dist-test-slave-6bbx"
server_key: "0cb1831e4bbf339c6ed44118c874d5ca"
server_key_iv: "50952497d09213753ac87783dec2c35a"
server_key_version: "encryptionkey@0"
I20260812 03:06:51.803947 1344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:06:51.824908 1344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:06:51.827744 1344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:06:51.827909 1344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:06:51.828510 1344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:06:51.829308 1344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:06:51.829365 1344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:06:51.829420 1344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:06:51.829445 1344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:06:51.837862 1344 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:36495
I20260812 03:06:51.837895 1536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:36495 every 8 connection(s)
I20260812 03:06:51.838784 1344 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:06:51.840519 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 1344
I20260812 03:06:51.842818 1539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:51.840952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.131:39301)
0812 03:06:51.841426 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:51.841550 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":127,"threads_started":1}
W20260812 03:06:51.843225 1537 heartbeater.cc:644] Failed to heartbeat to 127.5.96.190:37671 (0 consecutive failures): Network error: Failed to ping master at 127.5.96.190:37671: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
I20260812 03:06:51.843792 1539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:51.843545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.131:35791)
0812 03:06:51.843685 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:51.843738 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 03:06:51.844367 1539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:51.844169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.131:45613)
0812 03:06:51.844270 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:51.844316 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 03:06:51.844523 1537 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 03:06:52.541145 1546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:52.540740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.129:45851)
0812 03:06:52.540984 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:52.541035 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":89,"threads_started":1}
I20260812 03:06:52.659092 1547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:52.658715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.130:48889)
0812 03:06:52.658954 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:52.659002 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":67,"threads_started":1}
I20260812 03:06:52.845360 1548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:52.844991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.131:54451)
0812 03:06:52.845219 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:52.845266 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":103,"threads_started":1}
I20260812 03:06:52.953177 1550 raft_consensus.cc:482] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:06:52.953383 1550 raft_consensus.cc:504] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } }
I20260812 03:06:52.954449 1550 leader_election.cc:288] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84410a7f07064d2eb11893a02950535f (127.5.96.130:40785), ce9c4afdaee0491ba04d19e2b31fd28f (127.5.96.131:36495)
Aug 12 03:06:52 dist-test-slave-6bbx krb5kdc[31557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504011, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for kudu/127.5.96.130@KRBTEST.COM
Aug 12 03:06:52 dist-test-slave-6bbx krb5kdc[31557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504011, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for kudu/127.5.96.131@KRBTEST.COM
I20260812 03:06:52.969128 1551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:52.954671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:36495 (local address 127.5.96.129:56013)
0812 03:06:52.954943 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:52.954954 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:06:52.955048 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:06:52.956701 (+ 1653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:06:52.956707 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:06:52.957220 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:06:52.957676 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:52.957691 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:52.958948 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:52.958950 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:52.960262 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:52.960269 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:52.961191 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:52.961195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:52.961472 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:52.962151 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:06:52.962164 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:06:52.964411 (+ 2247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:06:52.966395 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:06:52.966398 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:06:52.966400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:06:52.966724 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:06:52.967097 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:06:52.967100 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:06:52.967102 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:06:52.967322 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:06:52.968058 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:06:52.968065 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:06:52.968409 (+ 344us) client_negotiation.cc:769] Sending connection context
0812 03:06:52.968618 (+ 209us) client_negotiation.cc:240] Negotiation successful
0812 03:06:52.968921 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":221,"spinlock_wait_cycles":26752,"thread_start_us":68,"threads_started":1}
I20260812 03:06:52.969205 1555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:52.954905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:56013 (local address 127.5.96.131:36495)
0812 03:06:52.955204 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0812 03:06:52.955230 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0812 03:06:52.955277 (+ 47us) server_negotiation.cc:407] Connection header received
0812 03:06:52.956012 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:06:52.956019 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:06:52.956353 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:06:52.956624 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:06:52.957848 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:52.958816 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:52.960375 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:52.961079 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:52.961239 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:52.964598 (+ 3359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:06:52.964604 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:06:52.964619 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:06:52.964651 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:06:52.966303 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:06:52.966835 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:06:52.966841 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:06:52.966847 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:06:52.967013 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:06:52.967409 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:06:52.967412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:06:52.967414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:06:52.967950 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:06:52.968083 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 03:06:52.968752 (+ 669us) server_negotiation.cc:299] Negotiation successful
0812 03:06:52.969024 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":104,"threads_started":1}
I20260812 03:06:52.969204 1552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:52.954780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:38321 (local address 127.5.96.130:40785)
0812 03:06:52.955259 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0812 03:06:52.955270 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:06:52.955302 (+ 32us) server_negotiation.cc:407] Connection header received
0812 03:06:52.956036 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:06:52.956043 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:06:52.956352 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:06:52.956624 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:06:52.957857 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:52.959080 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:52.960375 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:06:52.961079 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:06:52.961244 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:52.964313 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:06:52.964318 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:06:52.964328 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:06:52.964353 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:06:52.966186 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:06:52.966836 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:06:52.966841 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:06:52.966847 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:06:52.967008 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:06:52.967409 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:06:52.967412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:06:52.967414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:06:52.967950 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:06:52.968083 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 03:06:52.968742 (+ 659us) server_negotiation.cc:299] Negotiation successful
0812 03:06:52.969024 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":93,"threads_started":1}
I20260812 03:06:52.969130 1546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:52.954608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:40785 (local address 127.5.96.129:38321)
0812 03:06:52.954774 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:52.954787 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:06:52.955241 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:06:52.956701 (+ 1460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:06:52.956707 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:06:52.957230 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:06:52.957677 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:52.957692 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:52.959209 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:52.959212 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:52.960258 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:06:52.960269 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:06:52.961191 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:06:52.961195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:06:52.961472 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:06:52.962136 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:06:52.962157 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:06:52.964092 (+ 1935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:06:52.966310 (+ 2218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:06:52.966314 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:06:52.966326 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:06:52.966724 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:06:52.967099 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:06:52.967101 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:06:52.967103 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:06:52.967322 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:06:52.968058 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:06:52.968065 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:06:52.968409 (+ 344us) client_negotiation.cc:769] Sending connection context
0812 03:06:52.968618 (+ 209us) client_negotiation.cc:240] Negotiation successful
0812 03:06:52.968985 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":235,"spinlock_wait_cycles":48896}
W20260812 03:06:52.971758 934 leader_election.cc:341] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ce9c4afdaee0491ba04d19e2b31fd28f (127.5.96.131:36495): Not found: Tablet not found: f48ad560e26446c99c7765d57c3de980
W20260812 03:06:52.971913 930 leader_election.cc:341] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 84410a7f07064d2eb11893a02950535f (127.5.96.130:40785): Not found: Tablet not found: f48ad560e26446c99c7765d57c3de980
I20260812 03:06:52.971994 930 leader_election.cc:302] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cfcd528c8f4b40b5a86d56d912041895; no voters: 84410a7f07064d2eb11893a02950535f, ce9c4afdaee0491ba04d19e2b31fd28f
I20260812 03:06:52.972225 1550 raft_consensus.cc:2726] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:06:53.542222 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:53.541807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.129:49859)
0812 03:06:53.542054 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:53.542120 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":76,"threads_started":1}
I20260812 03:06:53.660089 1557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:53.659736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.130:50811)
0812 03:06:53.659955 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:53.660006 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":68,"threads_started":1}
I20260812 03:06:53.846297 1558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:53.845943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.131:42507)
0812 03:06:53.846150 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:53.846199 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":70,"threads_started":1}
I20260812 03:06:54.543241 1560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:54.542855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.129:47163)
0812 03:06:54.543093 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:54.543143 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":69,"threads_started":1}
I20260812 03:06:54.661089 1561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:54.660694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.130:48183)
0812 03:06:54.660953 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:54.661005 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":100,"threads_started":1}
I20260812 03:06:54.847285 1563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:54.846903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.131:35153)
0812 03:06:54.847151 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:54.847199 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":77,"threads_started":1}
I20260812 03:06:54.954023 1564 raft_consensus.cc:482] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:06:54.954195 1564 raft_consensus.cc:504] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } }
I20260812 03:06:54.954598 1564 leader_election.cc:288] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84410a7f07064d2eb11893a02950535f (127.5.96.130:40785), ce9c4afdaee0491ba04d19e2b31fd28f (127.5.96.131:36495)
W20260812 03:06:54.955391 934 leader_election.cc:341] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ce9c4afdaee0491ba04d19e2b31fd28f (127.5.96.131:36495): Not found: Tablet not found: f48ad560e26446c99c7765d57c3de980
W20260812 03:06:54.955520 930 leader_election.cc:341] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 84410a7f07064d2eb11893a02950535f (127.5.96.130:40785): Not found: Tablet not found: f48ad560e26446c99c7765d57c3de980
I20260812 03:06:54.955627 930 leader_election.cc:302] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cfcd528c8f4b40b5a86d56d912041895; no voters: 84410a7f07064d2eb11893a02950535f, ce9c4afdaee0491ba04d19e2b31fd28f
I20260812 03:06:54.955786 1564 raft_consensus.cc:2726] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:06:55.544274 1568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:55.543896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.129:33351)
0812 03:06:55.544119 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:55.544168 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":70,"threads_started":1}
I20260812 03:06:55.662048 1571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:55.661689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.130:56665)
0812 03:06:55.661918 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:55.661968 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":79,"threads_started":1}
I20260812 03:06:55.848332 1572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:55.847940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.131:51741)
0812 03:06:55.848183 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:55.848242 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":85,"threads_started":1}
I20260812 03:06:56.545471 1573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:56.545017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.129:52065)
0812 03:06:56.545320 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:56.545374 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":90,"threads_started":1}
I20260812 03:06:56.663039 1574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:56.662664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.130:39245)
0812 03:06:56.662901 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:56.662954 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":78,"threads_started":1}
I20260812 03:06:56.849349 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:56.848973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.131:49599)
0812 03:06:56.849208 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:56.849257 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":90,"threads_started":1}
I20260812 03:06:57.154215 1578 raft_consensus.cc:482] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:06:57.154348 1578 raft_consensus.cc:504] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } }
I20260812 03:06:57.154722 1578 leader_election.cc:288] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84410a7f07064d2eb11893a02950535f (127.5.96.130:40785), ce9c4afdaee0491ba04d19e2b31fd28f (127.5.96.131:36495)
W20260812 03:06:57.155454 930 leader_election.cc:341] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 84410a7f07064d2eb11893a02950535f (127.5.96.130:40785): Not found: Tablet not found: f48ad560e26446c99c7765d57c3de980
W20260812 03:06:57.155572 934 leader_election.cc:341] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ce9c4afdaee0491ba04d19e2b31fd28f (127.5.96.131:36495): Not found: Tablet not found: f48ad560e26446c99c7765d57c3de980
I20260812 03:06:57.155640 934 leader_election.cc:302] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cfcd528c8f4b40b5a86d56d912041895; no voters: 84410a7f07064d2eb11893a02950535f, ce9c4afdaee0491ba04d19e2b31fd28f
I20260812 03:06:57.155817 1578 raft_consensus.cc:2726] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:06:57.546502 1581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:57.546131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.129:50917)
0812 03:06:57.546352 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:57.546403 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":74,"threads_started":1}
I20260812 03:06:57.664012 1582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:57.663639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.130:45169)
0812 03:06:57.663880 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:57.663930 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":66,"threads_started":1}
I20260812 03:06:57.850317 1583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:57.849924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.131:58465)
0812 03:06:57.850167 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:57.850218 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":85,"threads_started":1}
I20260812 03:06:58.547518 1585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:58.547138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.129:60009)
0812 03:06:58.547366 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:58.547419 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":80,"threads_started":1}
I20260812 03:06:58.665038 1586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:58.664613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.130:35891)
0812 03:06:58.664899 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:58.664951 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":69,"threads_started":1}
I20260812 03:06:58.851310 1587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:58.850890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.131:34965)
0812 03:06:58.851170 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:58.851222 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":67,"threads_started":1}
I20260812 03:06:59.235759 1589 raft_consensus.cc:482] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:06:59.235889 1589 raft_consensus.cc:504] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } }
I20260812 03:06:59.236280 1589 leader_election.cc:288] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84410a7f07064d2eb11893a02950535f (127.5.96.130:40785), ce9c4afdaee0491ba04d19e2b31fd28f (127.5.96.131:36495)
W20260812 03:06:59.236943 930 leader_election.cc:341] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 84410a7f07064d2eb11893a02950535f (127.5.96.130:40785): Not found: Tablet not found: f48ad560e26446c99c7765d57c3de980
W20260812 03:06:59.237056 934 leader_election.cc:341] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ce9c4afdaee0491ba04d19e2b31fd28f (127.5.96.131:36495): Not found: Tablet not found: f48ad560e26446c99c7765d57c3de980
I20260812 03:06:59.237135 934 leader_election.cc:302] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cfcd528c8f4b40b5a86d56d912041895; no voters: 84410a7f07064d2eb11893a02950535f, ce9c4afdaee0491ba04d19e2b31fd28f
I20260812 03:06:59.237278 1589 raft_consensus.cc:2726] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:06:59.548589 1591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:59.548190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.129:44993)
0812 03:06:59.548439 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:59.548492 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":94,"threads_started":1}
I20260812 03:06:59.666057 1592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:59.665689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.130:48877)
0812 03:06:59.665919 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:59.665969 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":69,"threads_started":1}
I20260812 03:06:59.852350 1595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:06:59.851965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.131:58513)
0812 03:06:59.852194 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 03:06:59.852256 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":79,"threads_started":1}
I20260812 03:07:00.549647 1596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:00.549228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.129:56839)
0812 03:07:00.549490 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:00.549546 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":75,"threads_started":1}
I20260812 03:07:00.666982 1598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:00.666629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.130:53075)
0812 03:07:00.666849 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:00.666898 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":69,"threads_started":1}
I20260812 03:07:00.853325 1599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:00.852961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.131:44425)
0812 03:07:00.853187 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:00.853236 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":72,"threads_started":1}
I20260812 03:07:01.550828 1600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:01.550435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.129:36147)
0812 03:07:01.550676 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:01.550730 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":73,"threads_started":1}
I20260812 03:07:01.668000 1601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:01.667587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.130:44441)
0812 03:07:01.667858 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:01.667912 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":113,"threads_started":1}
I20260812 03:07:01.854339 1603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:01.853979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.131:45485)
0812 03:07:01.854196 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:01.854248 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":71,"threads_started":1}
I20260812 03:07:02.007656 1606 raft_consensus.cc:482] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:07:02.007829 1606 raft_consensus.cc:504] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84410a7f07064d2eb11893a02950535f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 40785 } } peers { permanent_uuid: "ce9c4afdaee0491ba04d19e2b31fd28f" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 36495 } } peers { permanent_uuid: "cfcd528c8f4b40b5a86d56d912041895" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 36745 } }
I20260812 03:07:02.008260 1606 leader_election.cc:288] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84410a7f07064d2eb11893a02950535f (127.5.96.130:40785), ce9c4afdaee0491ba04d19e2b31fd28f (127.5.96.131:36495)
W20260812 03:07:02.008982 934 leader_election.cc:341] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ce9c4afdaee0491ba04d19e2b31fd28f (127.5.96.131:36495): Not found: Tablet not found: f48ad560e26446c99c7765d57c3de980
W20260812 03:07:02.009091 930 leader_election.cc:341] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 84410a7f07064d2eb11893a02950535f (127.5.96.130:40785): Not found: Tablet not found: f48ad560e26446c99c7765d57c3de980
I20260812 03:07:02.009155 930 leader_election.cc:302] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cfcd528c8f4b40b5a86d56d912041895; no voters: 84410a7f07064d2eb11893a02950535f, ce9c4afdaee0491ba04d19e2b31fd28f
I20260812 03:07:02.009289 1606 raft_consensus.cc:2726] T f48ad560e26446c99c7765d57c3de980 P cfcd528c8f4b40b5a86d56d912041895 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:07:02.551869 1607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:02.551477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.129:34645)
0812 03:07:02.551720 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:02.551772 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":71,"threads_started":1}
I20260812 03:07:02.668998 1608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:02.668634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.130:54361)
0812 03:07:02.668858 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:02.668908 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":71,"threads_started":1}
I20260812 03:07:02.855367 1609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:02.854986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.131:41363)
0812 03:07:02.855224 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:02.855277 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":72,"threads_started":1}
I20260812 03:07:03.552877 1611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:03.552487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.129:33369)
0812 03:07:03.552729 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:03.552780 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":71,"threads_started":1}
I20260812 03:07:03.669948 1612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:03.669581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.130:50173)
0812 03:07:03.669813 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:03.669864 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":70,"threads_started":1}
I20260812 03:07:03.856395 1613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:03.855999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.131:42625)
0812 03:07:03.856250 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:03.856305 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":73,"threads_started":1}
I20260812 03:07:04.553932 1614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:04.553529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.129:42195)
0812 03:07:04.553779 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:04.553830 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":76,"threads_started":1}
I20260812 03:07:04.670919 1615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:04.670555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.130:58475)
0812 03:07:04.670786 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:04.670833 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":85,"threads_started":1}
I20260812 03:07:04.857376 1616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:04.857002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.131:53487)
0812 03:07:04.857221 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:04.857283 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":70,"threads_started":1}
I20260812 03:07:05.554956 1619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:05.554576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.129:52715)
0812 03:07:05.554808 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:05.554859 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":73,"threads_started":1}
I20260812 03:07:05.671881 1622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:05.671497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.130:42447)
0812 03:07:05.671739 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:05.671791 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":71,"threads_started":1}
I20260812 03:07:05.858361 1623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:05.857972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.131:49323)
0812 03:07:05.858207 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:05.858268 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":71,"threads_started":1}
I20260812 03:07:06.556181 1625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:06.555803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.129:57421)
0812 03:07:06.556032 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:06.556082 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":72,"threads_started":1}
I20260812 03:07:06.672927 1626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:06.672561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.130:42501)
0812 03:07:06.672790 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:06.672838 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":73,"threads_started":1}
I20260812 03:07:06.848984 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 845
I20260812 03:07:06.859568 1628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:06.859194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37671 (local address 127.5.96.131:53027)
0812 03:07:06.859427 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:06.859478 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.190:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":69,"threads_started":1}
I20260812 03:07:06.862742 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 1136
I20260812 03:07:06.874735 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 1344
2026-08-12T03:07:06Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59116 ms)
[----------] 30 tests from SecurityITest (236200 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:07:06 dist-test-slave-6bbx krb5kdc[1634](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:07:06 dist-test-slave-6bbx krb5kdc[1634](info): set up 2 sockets
Aug 12 03:07:06 dist-test-slave-6bbx krb5kdc[1634](info): commencing operation
krb5kdc: starting...
W20260812 03:07:08.030773 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:07:08 dist-test-slave-6bbx krb5kdc[1634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:07:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:07:08.139554 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:35719
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:45907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:35719
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:08.244979 1656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:08.245208 1656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:08.245244 1656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:08.249046 1656 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:08.249091 1656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:08.249109 1656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:08.249130 1656 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:08.249147 1656 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:08.254019 1656 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:45907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:35719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:35719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:08.254868 1656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:08.255770 1656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:08.263437 1670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:08.263478 1669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:08.263463 1672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:08.263772 1656 server_base.cc:1034] running on GCE node
I20260812 03:07:08.264293 1656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:08.265282 1656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:08.266439 1656 hybrid_clock.cc:648] HybridClock initialized: now 1786504028266427 us; error 27 us; skew 500 ppm
Aug 12 03:07:08 dist-test-slave-6bbx krb5kdc[1634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504028, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:08.268931 1656 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:07:08.269716 1656 webserver.cc:460] Webserver started at http://127.5.96.190:42627/ using document root <none> and password file <none>
I20260812 03:07:08.270202 1656 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:08.270275 1656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:08.270462 1656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:08.272359 1656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "855aee73f62f4c3fb666c56c9a412525"
format_stamp: "Formatted at 2026-08-12 03:07:08 on dist-test-slave-6bbx"
server_key: "d7b0ddfbcb29298007a2bcf1521a2a28"
server_key_iv: "83241932659f9826532b9658c70b9a8d"
server_key_version: "encryptionkey@0"
I20260812 03:07:08.272866 1656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "855aee73f62f4c3fb666c56c9a412525"
format_stamp: "Formatted at 2026-08-12 03:07:08 on dist-test-slave-6bbx"
server_key: "d7b0ddfbcb29298007a2bcf1521a2a28"
server_key_iv: "83241932659f9826532b9658c70b9a8d"
server_key_version: "encryptionkey@0"
I20260812 03:07:08.276206 1656 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:07:08.278331 1681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:08.279424 1656 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:07:08.279532 1656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "855aee73f62f4c3fb666c56c9a412525"
format_stamp: "Formatted at 2026-08-12 03:07:08 on dist-test-slave-6bbx"
server_key: "d7b0ddfbcb29298007a2bcf1521a2a28"
server_key_iv: "83241932659f9826532b9658c70b9a8d"
server_key_version: "encryptionkey@0"
I20260812 03:07:08.279618 1656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:08.292955 1656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:08.295317 1656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:08.295473 1656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:08.301952 1656 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:35719
I20260812 03:07:08.301998 1759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:35719 every 8 connection(s)
I20260812 03:07:08.302819 1656 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:08.304263 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 1656
I20260812 03:07:08.304389 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:07:08.304679 5506 external_mini_cluster.cc:1428] Setting key fd9af7d1e10303aa2d8896db78300002
I20260812 03:07:08.306033 1763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:08.311969 1763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 855aee73f62f4c3fb666c56c9a412525: Bootstrap starting.
Aug 12 03:07:08 dist-test-slave-6bbx krb5kdc[1634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:07:08.314373 1763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 855aee73f62f4c3fb666c56c9a412525: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:08.315248 1763 log.cc:824] T 00000000000000000000000000000000 P 855aee73f62f4c3fb666c56c9a412525: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:08.317333 1763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 855aee73f62f4c3fb666c56c9a412525: No bootstrap required, opened a new log
I20260812 03:07:08.318396 1767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:08.305910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56432 (local address 127.5.96.190:35719)
0812 03:07:08.306304 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:08.306320 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:08.306354 (+ 34us) server_negotiation.cc:407] Connection header received
0812 03:07:08.306958 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:08.306973 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:08.307271 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:08.307529 (+ 258us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:08.308414 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:08.309458 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:08.310574 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:08.311016 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:08.311123 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:08.313701 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:08.313707 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:08.313722 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:08.313752 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:08.315483 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:08.315993 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:08.315998 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:08.316004 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:08.316138 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:08.316399 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:08.316402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:08.316404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:08.316868 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:08.317048 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:08.317234 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 03:07:08.317517 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":132,"threads_started":1}
I20260812 03:07:08.320791 1763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 855aee73f62f4c3fb666c56c9a412525 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "855aee73f62f4c3fb666c56c9a412525" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35719 } }
I20260812 03:07:08.321030 1763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 855aee73f62f4c3fb666c56c9a412525 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:08.321075 1763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 855aee73f62f4c3fb666c56c9a412525 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 855aee73f62f4c3fb666c56c9a412525, State: Initialized, Role: FOLLOWER
I20260812 03:07:08.321404 1763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 855aee73f62f4c3fb666c56c9a412525 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "855aee73f62f4c3fb666c56c9a412525" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35719 } }
I20260812 03:07:08.321508 1763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 855aee73f62f4c3fb666c56c9a412525 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:07:08.321550 1763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 855aee73f62f4c3fb666c56c9a412525 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:07:08.321615 1763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 855aee73f62f4c3fb666c56c9a412525 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:08.322359 1763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 855aee73f62f4c3fb666c56c9a412525 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "855aee73f62f4c3fb666c56c9a412525" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35719 } }
I20260812 03:07:08.322631 1763 leader_election.cc:302] T 00000000000000000000000000000000 P 855aee73f62f4c3fb666c56c9a412525 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 855aee73f62f4c3fb666c56c9a412525; no voters:
I20260812 03:07:08.322876 1763 leader_election.cc:288] T 00000000000000000000000000000000 P 855aee73f62f4c3fb666c56c9a412525 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:07:08.322956 1772 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 855aee73f62f4c3fb666c56c9a412525 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:08.323148 1772 raft_consensus.cc:686] T 00000000000000000000000000000000 P 855aee73f62f4c3fb666c56c9a412525 [term 1 LEADER]: Becoming Leader. State: Replica: 855aee73f62f4c3fb666c56c9a412525, State: Running, Role: LEADER
I20260812 03:07:08.323478 1772 consensus_queue.cc:237] T 00000000000000000000000000000000 P 855aee73f62f4c3fb666c56c9a412525 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "855aee73f62f4c3fb666c56c9a412525" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35719 } }
I20260812 03:07:08.323963 1763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 855aee73f62f4c3fb666c56c9a412525 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:08.324996 1774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 855aee73f62f4c3fb666c56c9a412525 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 855aee73f62f4c3fb666c56c9a412525. Latest consensus state: current_term: 1 leader_uuid: "855aee73f62f4c3fb666c56c9a412525" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "855aee73f62f4c3fb666c56c9a412525" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35719 } } }
I20260812 03:07:08.325129 1774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 855aee73f62f4c3fb666c56c9a412525 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:08.325393 1773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 855aee73f62f4c3fb666c56c9a412525 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "855aee73f62f4c3fb666c56c9a412525" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "855aee73f62f4c3fb666c56c9a412525" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35719 } } }
I20260812 03:07:08.325472 1773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 855aee73f62f4c3fb666c56c9a412525 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:08.325505 1783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:08.328689 1783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:08.333724 1783 catalog_manager.cc:1347] Generated new cluster ID: 3503891d13604238b2234409847c983c
I20260812 03:07:08.333787 1783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:08.349826 1783 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:08.350641 1783 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:08.355686 1783 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 855aee73f62f4c3fb666c56c9a412525: Generated new TSK 0
I20260812 03:07:08.356249 1783 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.96.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
I20260812 03:07:08.392191 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:0
--local_ip_for_outbound_sockets=127.5.96.129
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:35719
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:45907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:08.500628 1805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:08.500845 1805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:08.500885 1805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:08.504751 1805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:08.504797 1805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:08.504849 1805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:07:08.509781 1805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:45907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:35719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:08.510605 1805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:08.511463 1805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:08.520257 1813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:08.520401 1815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:08.520427 1805 server_base.cc:1034] running on GCE node
W20260812 03:07:08.520268 1811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:08.520844 1805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:08.521411 1805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:08.522562 1805 hybrid_clock.cc:648] HybridClock initialized: now 1786504028522548 us; error 23 us; skew 500 ppm
Aug 12 03:07:08 dist-test-slave-6bbx krb5kdc[1634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504028, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:08.525154 1805 init.cc:377] Logged in from keytab as kudu/127.5.96.129@KRBTEST.COM (short username kudu)
I20260812 03:07:08.525967 1805 webserver.cc:460] Webserver started at http://127.5.96.129:44591/ using document root <none> and password file <none>
I20260812 03:07:08.526464 1805 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:08.526528 1805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:08.526702 1805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:08.528609 1805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-0/data/instance:
uuid: "197d9e0fbc8c45e1a1cccf0ae99f2b32"
format_stamp: "Formatted at 2026-08-12 03:07:08 on dist-test-slave-6bbx"
server_key: "2eec44fd7427c323578bafa522e13938"
server_key_iv: "2c600cd5865be1d58f0572da46181026"
server_key_version: "encryptionkey@0"
I20260812 03:07:08.529120 1805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance:
uuid: "197d9e0fbc8c45e1a1cccf0ae99f2b32"
format_stamp: "Formatted at 2026-08-12 03:07:08 on dist-test-slave-6bbx"
server_key: "2eec44fd7427c323578bafa522e13938"
server_key_iv: "2c600cd5865be1d58f0572da46181026"
server_key_version: "encryptionkey@0"
I20260812 03:07:08.532475 1805 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 03:07:08.534582 1827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:08.535635 1805 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:07:08.535781 1805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "197d9e0fbc8c45e1a1cccf0ae99f2b32"
format_stamp: "Formatted at 2026-08-12 03:07:08 on dist-test-slave-6bbx"
server_key: "2eec44fd7427c323578bafa522e13938"
server_key_iv: "2c600cd5865be1d58f0572da46181026"
server_key_version: "encryptionkey@0"
I20260812 03:07:08.535871 1805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:08.553269 1805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:08.555622 1805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:08.555791 1805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:08.556290 1805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:07:08.557001 1805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:07:08.557047 1805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:08.557096 1805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:07:08.557125 1805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:08.565096 1805 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:34235
I20260812 03:07:08.565150 2002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:34235 every 8 connection(s)
I20260812 03:07:08.565850 1805 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:07:08.567698 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 1805
I20260812 03:07:08.567826 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance
I20260812 03:07:08.568076 5506 external_mini_cluster.cc:1428] Setting key 04c66ed75e0de9097da1858f08cb1312
Aug 12 03:07:08 dist-test-slave-6bbx krb5kdc[1634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504028, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.96.130@KRBTEST.COM; defaulting to no policy
I20260812 03:07:08.578647 1767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:08.567359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:44247 (local address 127.5.96.190:35719)
0812 03:07:08.567484 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:08.567489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:08.568275 (+ 786us) server_negotiation.cc:407] Connection header received
0812 03:07:08.569699 (+ 1424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:08.569702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:08.569755 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:08.569840 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:08.570880 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:08.571602 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:08.572719 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:08.572958 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:08.573042 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:08.575374 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:08.575376 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:08.575378 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:08.575400 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:08.576675 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:08.577234 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:08.577236 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:08.577237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:08.577306 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:08.577650 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:08.577652 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:08.577653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:08.577804 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:08.577871 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:08.578432 (+ 561us) server_negotiation.cc:299] Negotiation successful
0812 03:07:08.578531 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:07:08.579375 2006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:08.567639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:35719 (local address 127.5.96.129:44247)
0812 03:07:08.568118 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:08.568167 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:08.569508 (+ 1341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:08.569964 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:08.569970 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:08.570308 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:08.570724 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:08.570736 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:08.571748 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:08.571752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:08.572603 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:08.572613 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:08.573092 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:08.573096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:08.573252 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:08.573740 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:08.573755 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:08.575202 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:08.576790 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:08.576796 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:08.576809 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:08.577131 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:08.577388 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:08.577391 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:08.577393 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:08.577571 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:08.577896 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:08.577903 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:08.578201 (+ 298us) client_negotiation.cc:769] Sending connection context
0812 03:07:08.578376 (+ 175us) client_negotiation.cc:240] Negotiation successful
0812 03:07:08.578599 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":121,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.130@KRBTEST.COM" created.
I20260812 03:07:08.580528 2004 heartbeater.cc:347] Connected to a master server at 127.5.96.190:35719
I20260812 03:07:08.580740 2004 heartbeater.cc:464] Registering TS with master...
I20260812 03:07:08.581158 2004 heartbeater.cc:511] Master 127.5.96.190:35719 requested a full tablet report, sending...
I20260812 03:07:08.582647 1703 ts_manager.cc:194] Registered new tserver with Master: 197d9e0fbc8c45e1a1cccf0ae99f2b32 (127.5.96.129:34235)
I20260812 03:07:08.583832 1703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.129@KRBTEST.COM'} at 127.5.96.129:44247
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
I20260812 03:07:08.592422 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:0
--local_ip_for_outbound_sockets=127.5.96.130
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:35719
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:45907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:08.697751 2010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:08.698037 2010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:08.698097 2010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:08.702014 2010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:08.702064 2010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:08.702157 2010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:07:08.707139 2010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:45907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:35719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:08.708063 2010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:08.708971 2010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:08.717695 2018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:08.717686 2019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:08.717686 2024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:08.717768 2010 server_base.cc:1034] running on GCE node
I20260812 03:07:08.718312 2010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:08.718869 2010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:08.720028 2010 hybrid_clock.cc:648] HybridClock initialized: now 1786504028720014 us; error 23 us; skew 500 ppm
Aug 12 03:07:08 dist-test-slave-6bbx krb5kdc[1634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504028, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:08.722558 2010 init.cc:377] Logged in from keytab as kudu/127.5.96.130@KRBTEST.COM (short username kudu)
I20260812 03:07:08.723456 2010 webserver.cc:460] Webserver started at http://127.5.96.130:34199/ using document root <none> and password file <none>
I20260812 03:07:08.724014 2010 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:08.724077 2010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:08.724278 2010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:08.726243 2010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-1/data/instance:
uuid: "3f9d602a2a0e4b0798b007d031da0dd3"
format_stamp: "Formatted at 2026-08-12 03:07:08 on dist-test-slave-6bbx"
server_key: "7107c97e19a580a80ce1aa01f61bbdf4"
server_key_iv: "9ee0c60030fbd56fd888bafc4423be5b"
server_key_version: "encryptionkey@0"
I20260812 03:07:08.726771 2010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance:
uuid: "3f9d602a2a0e4b0798b007d031da0dd3"
format_stamp: "Formatted at 2026-08-12 03:07:08 on dist-test-slave-6bbx"
server_key: "7107c97e19a580a80ce1aa01f61bbdf4"
server_key_iv: "9ee0c60030fbd56fd888bafc4423be5b"
server_key_version: "encryptionkey@0"
I20260812 03:07:08.730218 2010 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260812 03:07:08.732434 2034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:08.733577 2010 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:07:08.733680 2010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "3f9d602a2a0e4b0798b007d031da0dd3"
format_stamp: "Formatted at 2026-08-12 03:07:08 on dist-test-slave-6bbx"
server_key: "7107c97e19a580a80ce1aa01f61bbdf4"
server_key_iv: "9ee0c60030fbd56fd888bafc4423be5b"
server_key_version: "encryptionkey@0"
I20260812 03:07:08.733769 2010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:08.745711 2010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:08.748128 2010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:08.748281 2010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:08.748790 2010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:07:08.749539 2010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:07:08.749585 2010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:08.749625 2010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:07:08.749653 2010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:08.757718 2010 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:36701
I20260812 03:07:08.757745 2213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:36701 every 8 connection(s)
I20260812 03:07:08.758478 2010 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
Aug 12 03:07:08 dist-test-slave-6bbx krb5kdc[1634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504028, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:07:08.767644 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 2010
I20260812 03:07:08.767791 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance
I20260812 03:07:08.768085 5506 external_mini_cluster.cc:1428] Setting key 5b2de354338faa8226cb802bdc3197de
I20260812 03:07:08.771145 1767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:08.760074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:32939 (local address 127.5.96.190:35719)
0812 03:07:08.760233 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:08.760237 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:08.760946 (+ 709us) server_negotiation.cc:407] Connection header received
0812 03:07:08.762195 (+ 1249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:08.762198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:08.762261 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:08.762331 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:08.763396 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:08.763952 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:08.765048 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:08.765275 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:08.765363 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:08.767656 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:08.767682 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:08.767685 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:08.767714 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:08.769180 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:08.769724 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:08.769727 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:08.769728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:08.769805 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:08.770174 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:08.770177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:08.770178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:08.770348 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:08.770423 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:08.770940 (+ 517us) server_negotiation.cc:299] Negotiation successful
0812 03:07:08.771028 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 03:07:08.772076 2219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:08.760396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:35719 (local address 127.5.96.130:32939)
0812 03:07:08.760813 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:08.760853 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:08.761989 (+ 1136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:08.762496 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:08.762502 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:08.762833 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:08.763258 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:08.763271 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:08.764073 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:08.764076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:08.764930 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:08.764939 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:08.765411 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:08.765415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:08.765571 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:08.766040 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:08.766055 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:08.767458 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:08.769313 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:08.769319 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:08.769332 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:08.769628 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:08.769893 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:08.769897 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:08.769899 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:08.770077 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:08.770445 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:08.770451 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:08.770727 (+ 276us) client_negotiation.cc:769] Sending connection context
0812 03:07:08.770916 (+ 189us) client_negotiation.cc:240] Negotiation successful
0812 03:07:08.771204 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":100,"threads_started":1}
I20260812 03:07:08.773240 2214 heartbeater.cc:347] Connected to a master server at 127.5.96.190:35719
I20260812 03:07:08.773460 2214 heartbeater.cc:464] Registering TS with master...
I20260812 03:07:08.773942 2214 heartbeater.cc:511] Master 127.5.96.190:35719 requested a full tablet report, sending...
I20260812 03:07:08.775000 1703 ts_manager.cc:194] Registered new tserver with Master: 3f9d602a2a0e4b0798b007d031da0dd3 (127.5.96.130:36701)
I20260812 03:07:08.775650 1703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.130@KRBTEST.COM'} at 127.5.96.130:32939
WARNING: no policy specified for kudu/127.5.96.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
I20260812 03:07:08.793805 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:0
--local_ip_for_outbound_sockets=127.5.96.131
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:35719
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:45907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:08.900641 2223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:08.900867 2223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:08.900919 2223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:08.904791 2223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:08.904835 2223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:08.904888 2223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:07:08.909723 2223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:45907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:35719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:08.910565 2223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:08.911437 2223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:08.920385 2236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:08.920514 2239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:08.920378 2234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:08.920430 2223 server_base.cc:1034] running on GCE node
I20260812 03:07:08.920935 2223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:08.921482 2223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:08.922633 2223 hybrid_clock.cc:648] HybridClock initialized: now 1786504028922616 us; error 27 us; skew 500 ppm
Aug 12 03:07:08 dist-test-slave-6bbx krb5kdc[1634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504028, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:08.925323 2223 init.cc:377] Logged in from keytab as kudu/127.5.96.131@KRBTEST.COM (short username kudu)
I20260812 03:07:08.926131 2223 webserver.cc:460] Webserver started at http://127.5.96.131:38819/ using document root <none> and password file <none>
I20260812 03:07:08.926617 2223 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:08.926697 2223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:08.926889 2223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:08.928858 2223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-2/data/instance:
uuid: "b6150724bd8340d1b20cbe1399709b37"
format_stamp: "Formatted at 2026-08-12 03:07:08 on dist-test-slave-6bbx"
server_key: "2a5ce19fec85126e5873e37821e59778"
server_key_iv: "939bf37109d4f6edd06bd16751daf073"
server_key_version: "encryptionkey@0"
I20260812 03:07:08.929376 2223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance:
uuid: "b6150724bd8340d1b20cbe1399709b37"
format_stamp: "Formatted at 2026-08-12 03:07:08 on dist-test-slave-6bbx"
server_key: "2a5ce19fec85126e5873e37821e59778"
server_key_iv: "939bf37109d4f6edd06bd16751daf073"
server_key_version: "encryptionkey@0"
I20260812 03:07:08.932775 2223 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:07:08.934921 2252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:08.936172 2223 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:07:08.936272 2223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "b6150724bd8340d1b20cbe1399709b37"
format_stamp: "Formatted at 2026-08-12 03:07:08 on dist-test-slave-6bbx"
server_key: "2a5ce19fec85126e5873e37821e59778"
server_key_iv: "939bf37109d4f6edd06bd16751daf073"
server_key_version: "encryptionkey@0"
I20260812 03:07:08.936357 2223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:08.946923 2223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:08.949337 2223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:08.949487 2223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:08.949990 2223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:07:08.950711 2223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:07:08.950757 2223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:08.950805 2223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:07:08.950834 2223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:08.958909 2223 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:45225
I20260812 03:07:08.958962 2425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:45225 every 8 connection(s)
I20260812 03:07:08.959712 2223 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:07:08 dist-test-slave-6bbx krb5kdc[1634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504028, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:07:08.969033 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 2223
I20260812 03:07:08.969157 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance
I20260812 03:07:08.969432 5506 external_mini_cluster.cc:1428] Setting key 0076cbb5c6af38447259c9520bcfbd52
I20260812 03:07:08.972108 1767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:08.961304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:39871 (local address 127.5.96.190:35719)
0812 03:07:08.961462 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:08.961467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:08.962217 (+ 750us) server_negotiation.cc:407] Connection header received
0812 03:07:08.963433 (+ 1216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:08.963436 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:08.963482 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:08.963550 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:08.964613 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:08.965140 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:08.966176 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:08.966376 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:08.966440 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:08.968723 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:08.968726 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:08.968728 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:08.968764 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:08.970157 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:08.970652 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:08.970656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:08.970658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:08.970742 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:08.971046 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:08.971050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:08.971051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:08.971279 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:08.971377 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:08.971897 (+ 520us) server_negotiation.cc:299] Negotiation successful
0812 03:07:08.971996 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 03:07:08.972801 2428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:08.961637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:35719 (local address 127.5.96.131:39871)
0812 03:07:08.962064 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:08.962118 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:08.963254 (+ 1136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:08.963731 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:08.963740 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:08.964075 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:08.964506 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:08.964519 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:08.965239 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:08.965242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:08.966079 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:08.966088 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:08.966475 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:08.966478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:08.966614 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:08.967126 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:08.967143 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:08.968569 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:08.970294 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:08.970300 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:08.970313 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:08.970561 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:08.970831 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:08.970833 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:08.970834 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:08.970966 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:08.971411 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:08.971418 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:08.971652 (+ 234us) client_negotiation.cc:769] Sending connection context
0812 03:07:08.971851 (+ 199us) client_negotiation.cc:240] Negotiation successful
0812 03:07:08.972081 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":105,"threads_started":1}
I20260812 03:07:08.973767 2426 heartbeater.cc:347] Connected to a master server at 127.5.96.190:35719
I20260812 03:07:08.973953 2426 heartbeater.cc:464] Registering TS with master...
I20260812 03:07:08.974354 2426 heartbeater.cc:511] Master 127.5.96.190:35719 requested a full tablet report, sending...
I20260812 03:07:08.975226 1703 ts_manager.cc:194] Registered new tserver with Master: b6150724bd8340d1b20cbe1399709b37 (127.5.96.131:45225)
I20260812 03:07:08.975795 1703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.131@KRBTEST.COM'} at 127.5.96.131:39871
I20260812 03:07:08.982926 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:07:08.990844 1767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:08.984813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56434 (local address 127.5.96.190:35719)
0812 03:07:08.984921 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:08.984924 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:08.985144 (+ 220us) server_negotiation.cc:407] Connection header received
0812 03:07:08.985185 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:08.985187 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:08.985225 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:08.985291 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:08.985879 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:08.986373 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:08.987111 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:08.987306 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:08.987371 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:08.988486 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:08.988488 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:08.988490 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:08.988512 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:08.989622 (+ 1110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:08.989968 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:08.989970 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:08.989972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:08.990039 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:08.990231 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:08.990233 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:08.990234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:08.990413 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:08.990511 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:08.990659 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 03:07:08.990741 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:07:08.993381 1703 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56434:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:07:08.996414 1703 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:07:09.009399 2451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:09.004591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57122 (local address 127.5.96.130:36701)
0812 03:07:09.005138 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:09.005150 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:09.005171 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:07:09.005244 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:09.005248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:09.005429 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:09.005576 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:09.005877 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.006945 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.008171 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.008741 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.008840 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:09.008981 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:09.009135 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 03:07:09.009239 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":103,"threads_started":1}
I20260812 03:07:09.009442 2450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:09.004008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:36701 (local address 127.0.0.1:57122)
0812 03:07:09.004722 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:09.004735 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:09.004820 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:09.005657 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:09.005660 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:09.005675 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:09.005753 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:09.005759 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.007059 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.007063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:09.008031 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:09.008041 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.008856 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.008859 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:09.008961 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:09.008997 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 03:07:09.009097 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 03:07:09.009325 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"mutex_wait_us":1,"thread_start_us":100,"threads_started":1}
I20260812 03:07:09.009794 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:09.003818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:34235 (local address 127.0.0.1:45426)
0812 03:07:09.004374 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:09.004422 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:09.004567 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:09.005359 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:09.005365 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:09.005387 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:09.005516 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:09.005524 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.006837 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.006840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:09.008118 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:09.008129 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.009410 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.009414 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:09.009504 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:09.009530 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 03:07:09.009608 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 03:07:09.009673 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":77,"threads_started":1}
I20260812 03:07:09.009910 2449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:09.003972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45426 (local address 127.5.96.129:34235)
0812 03:07:09.004422 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:09.004434 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:09.004731 (+ 297us) server_negotiation.cc:407] Connection header received
0812 03:07:09.004815 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:09.004820 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:09.005024 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:09.005179 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:09.005620 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.006702 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.008260 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.009119 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.009216 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:09.009344 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:09.009660 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 03:07:09.009766 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":140,"threads_started":1}
I20260812 03:07:09.010973 2447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:09.004496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:45225 (local address 127.0.0.1:47786)
0812 03:07:09.005829 (+ 1333us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:09.005841 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:09.005931 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:09.006277 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:09.006280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:09.006295 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:09.006393 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:09.006401 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.007639 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.007643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:09.008746 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:09.008756 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.010492 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.010495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:09.010688 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:09.010712 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:07:09.010793 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 03:07:09.010859 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1274,"thread_start_us":82,"threads_started":1}
I20260812 03:07:09.011587 2453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:09.004874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47786 (local address 127.5.96.131:45225)
0812 03:07:09.005230 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:09.005242 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:09.005931 (+ 689us) server_negotiation.cc:407] Connection header received
0812 03:07:09.006025 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:09.006030 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:09.006161 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:09.006282 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:09.006498 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.007457 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.008883 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.010188 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.010294 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:09.010413 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:09.011356 (+ 943us) server_negotiation.cc:299] Negotiation successful
0812 03:07:09.011448 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":84,"threads_started":1}
I20260812 03:07:09.012458 2117 tablet_service.cc:1467] Processing CreateTablet for tablet a0145646b06845229ef620fa11d0e78b (DEFAULT_TABLE table=test-table [id=8f1076c422f24e41b0489e725739b755]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:07:09.012682 1908 tablet_service.cc:1467] Processing CreateTablet for tablet a0145646b06845229ef620fa11d0e78b (DEFAULT_TABLE table=test-table [id=8f1076c422f24e41b0489e725739b755]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:07:09.013406 2117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0145646b06845229ef620fa11d0e78b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:09.013715 1908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0145646b06845229ef620fa11d0e78b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:09.014210 2321 tablet_service.cc:1467] Processing CreateTablet for tablet a0145646b06845229ef620fa11d0e78b (DEFAULT_TABLE table=test-table [id=8f1076c422f24e41b0489e725739b755]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:07:09.015170 2321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0145646b06845229ef620fa11d0e78b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:09.019794 2455 tablet_bootstrap.cc:492] T a0145646b06845229ef620fa11d0e78b P 3f9d602a2a0e4b0798b007d031da0dd3: Bootstrap starting.
I20260812 03:07:09.020535 2456 tablet_bootstrap.cc:492] T a0145646b06845229ef620fa11d0e78b P 197d9e0fbc8c45e1a1cccf0ae99f2b32: Bootstrap starting.
I20260812 03:07:09.021721 2457 tablet_bootstrap.cc:492] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37: Bootstrap starting.
I20260812 03:07:09.022437 2455 tablet_bootstrap.cc:654] T a0145646b06845229ef620fa11d0e78b P 3f9d602a2a0e4b0798b007d031da0dd3: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:09.022795 2456 tablet_bootstrap.cc:654] T a0145646b06845229ef620fa11d0e78b P 197d9e0fbc8c45e1a1cccf0ae99f2b32: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:09.023325 2455 log.cc:824] T a0145646b06845229ef620fa11d0e78b P 3f9d602a2a0e4b0798b007d031da0dd3: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:09.023598 2457 tablet_bootstrap.cc:654] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:09.024125 2456 log.cc:824] T a0145646b06845229ef620fa11d0e78b P 197d9e0fbc8c45e1a1cccf0ae99f2b32: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:09.024753 2457 log.cc:824] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:09.025383 2455 tablet_bootstrap.cc:492] T a0145646b06845229ef620fa11d0e78b P 3f9d602a2a0e4b0798b007d031da0dd3: No bootstrap required, opened a new log
I20260812 03:07:09.025576 2455 ts_tablet_manager.cc:1397] T a0145646b06845229ef620fa11d0e78b P 3f9d602a2a0e4b0798b007d031da0dd3: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.003s
I20260812 03:07:09.026111 2456 tablet_bootstrap.cc:492] T a0145646b06845229ef620fa11d0e78b P 197d9e0fbc8c45e1a1cccf0ae99f2b32: No bootstrap required, opened a new log
I20260812 03:07:09.026291 2456 ts_tablet_manager.cc:1397] T a0145646b06845229ef620fa11d0e78b P 197d9e0fbc8c45e1a1cccf0ae99f2b32: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 03:07:09.026510 2457 tablet_bootstrap.cc:492] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37: No bootstrap required, opened a new log
I20260812 03:07:09.026681 2457 ts_tablet_manager.cc:1397] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 03:07:09.029274 2455 raft_consensus.cc:348] T a0145646b06845229ef620fa11d0e78b P 3f9d602a2a0e4b0798b007d031da0dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197d9e0fbc8c45e1a1cccf0ae99f2b32" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34235 } } peers { permanent_uuid: "3f9d602a2a0e4b0798b007d031da0dd3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 36701 } } peers { permanent_uuid: "b6150724bd8340d1b20cbe1399709b37" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45225 } }
I20260812 03:07:09.029542 2455 raft_consensus.cc:374] T a0145646b06845229ef620fa11d0e78b P 3f9d602a2a0e4b0798b007d031da0dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:09.029611 2455 raft_consensus.cc:729] T a0145646b06845229ef620fa11d0e78b P 3f9d602a2a0e4b0798b007d031da0dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f9d602a2a0e4b0798b007d031da0dd3, State: Initialized, Role: FOLLOWER
I20260812 03:07:09.029462 2457 raft_consensus.cc:348] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197d9e0fbc8c45e1a1cccf0ae99f2b32" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34235 } } peers { permanent_uuid: "3f9d602a2a0e4b0798b007d031da0dd3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 36701 } } peers { permanent_uuid: "b6150724bd8340d1b20cbe1399709b37" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45225 } }
I20260812 03:07:09.029717 2457 raft_consensus.cc:374] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:09.029785 2457 raft_consensus.cc:729] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6150724bd8340d1b20cbe1399709b37, State: Initialized, Role: FOLLOWER
I20260812 03:07:09.029846 2456 raft_consensus.cc:348] T a0145646b06845229ef620fa11d0e78b P 197d9e0fbc8c45e1a1cccf0ae99f2b32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197d9e0fbc8c45e1a1cccf0ae99f2b32" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34235 } } peers { permanent_uuid: "3f9d602a2a0e4b0798b007d031da0dd3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 36701 } } peers { permanent_uuid: "b6150724bd8340d1b20cbe1399709b37" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45225 } }
I20260812 03:07:09.030081 2456 raft_consensus.cc:374] T a0145646b06845229ef620fa11d0e78b P 197d9e0fbc8c45e1a1cccf0ae99f2b32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:09.030061 2455 consensus_queue.cc:260] T a0145646b06845229ef620fa11d0e78b P 3f9d602a2a0e4b0798b007d031da0dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197d9e0fbc8c45e1a1cccf0ae99f2b32" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34235 } } peers { permanent_uuid: "3f9d602a2a0e4b0798b007d031da0dd3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 36701 } } peers { permanent_uuid: "b6150724bd8340d1b20cbe1399709b37" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45225 } }
I20260812 03:07:09.030138 2456 raft_consensus.cc:729] T a0145646b06845229ef620fa11d0e78b P 197d9e0fbc8c45e1a1cccf0ae99f2b32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 197d9e0fbc8c45e1a1cccf0ae99f2b32, State: Initialized, Role: FOLLOWER
I20260812 03:07:09.030189 2457 consensus_queue.cc:260] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197d9e0fbc8c45e1a1cccf0ae99f2b32" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34235 } } peers { permanent_uuid: "3f9d602a2a0e4b0798b007d031da0dd3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 36701 } } peers { permanent_uuid: "b6150724bd8340d1b20cbe1399709b37" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45225 } }
I20260812 03:07:09.030555 2456 consensus_queue.cc:260] T a0145646b06845229ef620fa11d0e78b P 197d9e0fbc8c45e1a1cccf0ae99f2b32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197d9e0fbc8c45e1a1cccf0ae99f2b32" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34235 } } peers { permanent_uuid: "3f9d602a2a0e4b0798b007d031da0dd3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 36701 } } peers { permanent_uuid: "b6150724bd8340d1b20cbe1399709b37" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45225 } }
I20260812 03:07:09.030845 2426 heartbeater.cc:503] Master 127.5.96.190:35719 was elected leader, sending a full tablet report...
I20260812 03:07:09.031013 2455 ts_tablet_manager.cc:1428] T a0145646b06845229ef620fa11d0e78b P 3f9d602a2a0e4b0798b007d031da0dd3: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260812 03:07:09.031097 2457 ts_tablet_manager.cc:1428] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260812 03:07:09.031322 2214 heartbeater.cc:503] Master 127.5.96.190:35719 was elected leader, sending a full tablet report...
I20260812 03:07:09.031378 2456 ts_tablet_manager.cc:1428] T a0145646b06845229ef620fa11d0e78b P 197d9e0fbc8c45e1a1cccf0ae99f2b32: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 03:07:09.031701 2004 heartbeater.cc:503] Master 127.5.96.190:35719 was elected leader, sending a full tablet report...
W20260812 03:07:09.066828 2005 tablet.cc:2369] T a0145646b06845229ef620fa11d0e78b P 197d9e0fbc8c45e1a1cccf0ae99f2b32: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:07:09.194875 2463 raft_consensus.cc:482] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:07:09.195075 2463 raft_consensus.cc:504] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197d9e0fbc8c45e1a1cccf0ae99f2b32" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34235 } } peers { permanent_uuid: "3f9d602a2a0e4b0798b007d031da0dd3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 36701 } } peers { permanent_uuid: "b6150724bd8340d1b20cbe1399709b37" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45225 } }
I20260812 03:07:09.196271 2463 leader_election.cc:288] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 197d9e0fbc8c45e1a1cccf0ae99f2b32 (127.5.96.129:34235), 3f9d602a2a0e4b0798b007d031da0dd3 (127.5.96.130:36701)
I20260812 03:07:09.199772 2451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:09.196652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:46443 (local address 127.5.96.130:36701)
0812 03:07:09.196800 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:09.196804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:09.196819 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:07:09.196913 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:09.196915 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:09.196955 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:09.197015 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:09.197258 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.197755 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.198825 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.199217 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.199325 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:09.199380 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:09.199558 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 03:07:09.199627 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 03:07:09.199765 2428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:09.196508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:36701 (local address 127.5.96.131:46443)
0812 03:07:09.196708 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:09.196724 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:09.196812 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:09.197034 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:09.197037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:09.197054 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:09.197157 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:09.197163 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.197858 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.197861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:09.198690 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:09.198701 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.199325 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.199329 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:09.199452 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:09.199471 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 03:07:09.199534 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 03:07:09.199603 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":23}
I20260812 03:07:09.200129 2449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:09.196594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:53277 (local address 127.5.96.129:34235)
0812 03:07:09.196721 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:09.196725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:09.197012 (+ 287us) server_negotiation.cc:407] Connection header received
0812 03:07:09.197133 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:09.197136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:09.197176 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:09.197238 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:09.197498 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.197984 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.199178 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.199582 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.199706 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:09.199765 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:09.199948 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 03:07:09.200018 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 03:07:09.200397 2150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0145646b06845229ef620fa11d0e78b" candidate_uuid: "b6150724bd8340d1b20cbe1399709b37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f9d602a2a0e4b0798b007d031da0dd3" is_pre_election: true
I20260812 03:07:09.200448 2467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:09.196508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:34235 (local address 127.5.96.131:53277)
0812 03:07:09.196922 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:09.196933 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:09.197033 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:09.197260 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:09.197263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:09.197278 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:09.197396 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:09.197401 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.198086 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.198089 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:09.199056 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:09.199066 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.199706 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.199710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:09.199805 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:09.199827 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:07:09.200271 (+ 444us) client_negotiation.cc:240] Negotiation successful
0812 03:07:09.200343 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":48,"thread_start_us":151,"threads_started":1}
I20260812 03:07:09.200690 2150 raft_consensus.cc:2454] T a0145646b06845229ef620fa11d0e78b P 3f9d602a2a0e4b0798b007d031da0dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6150724bd8340d1b20cbe1399709b37 in term 0.
I20260812 03:07:09.201057 1937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0145646b06845229ef620fa11d0e78b" candidate_uuid: "b6150724bd8340d1b20cbe1399709b37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197d9e0fbc8c45e1a1cccf0ae99f2b32" is_pre_election: true
I20260812 03:07:09.201167 2253 leader_election.cc:302] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f9d602a2a0e4b0798b007d031da0dd3, b6150724bd8340d1b20cbe1399709b37; no voters:
I20260812 03:07:09.201311 1937 raft_consensus.cc:2454] T a0145646b06845229ef620fa11d0e78b P 197d9e0fbc8c45e1a1cccf0ae99f2b32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6150724bd8340d1b20cbe1399709b37 in term 0.
I20260812 03:07:09.201442 2463 raft_consensus.cc:2781] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:07:09.201520 2463 raft_consensus.cc:482] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:07:09.201572 2463 raft_consensus.cc:3037] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:09.202620 2463 raft_consensus.cc:504] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197d9e0fbc8c45e1a1cccf0ae99f2b32" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34235 } } peers { permanent_uuid: "3f9d602a2a0e4b0798b007d031da0dd3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 36701 } } peers { permanent_uuid: "b6150724bd8340d1b20cbe1399709b37" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45225 } }
I20260812 03:07:09.202908 2463 leader_election.cc:288] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37 [CANDIDATE]: Term 1 election: Requested vote from peers 197d9e0fbc8c45e1a1cccf0ae99f2b32 (127.5.96.129:34235), 3f9d602a2a0e4b0798b007d031da0dd3 (127.5.96.130:36701)
I20260812 03:07:09.203243 1937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0145646b06845229ef620fa11d0e78b" candidate_uuid: "b6150724bd8340d1b20cbe1399709b37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197d9e0fbc8c45e1a1cccf0ae99f2b32"
I20260812 03:07:09.203318 2150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0145646b06845229ef620fa11d0e78b" candidate_uuid: "b6150724bd8340d1b20cbe1399709b37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f9d602a2a0e4b0798b007d031da0dd3"
I20260812 03:07:09.203361 1937 raft_consensus.cc:3037] T a0145646b06845229ef620fa11d0e78b P 197d9e0fbc8c45e1a1cccf0ae99f2b32 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:09.203441 2150 raft_consensus.cc:3037] T a0145646b06845229ef620fa11d0e78b P 3f9d602a2a0e4b0798b007d031da0dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:09.204411 1937 raft_consensus.cc:2454] T a0145646b06845229ef620fa11d0e78b P 197d9e0fbc8c45e1a1cccf0ae99f2b32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6150724bd8340d1b20cbe1399709b37 in term 1.
I20260812 03:07:09.204510 2150 raft_consensus.cc:2454] T a0145646b06845229ef620fa11d0e78b P 3f9d602a2a0e4b0798b007d031da0dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6150724bd8340d1b20cbe1399709b37 in term 1.
I20260812 03:07:09.204708 2257 leader_election.cc:302] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197d9e0fbc8c45e1a1cccf0ae99f2b32, b6150724bd8340d1b20cbe1399709b37; no voters:
I20260812 03:07:09.204872 2463 raft_consensus.cc:2781] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:09.205082 2463 raft_consensus.cc:686] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37 [term 1 LEADER]: Becoming Leader. State: Replica: b6150724bd8340d1b20cbe1399709b37, State: Running, Role: LEADER
I20260812 03:07:09.205394 2463 consensus_queue.cc:237] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197d9e0fbc8c45e1a1cccf0ae99f2b32" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34235 } } peers { permanent_uuid: "3f9d602a2a0e4b0798b007d031da0dd3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 36701 } } peers { permanent_uuid: "b6150724bd8340d1b20cbe1399709b37" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45225 } }
I20260812 03:07:09.208294 1700 catalog_manager.cc:5495] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37 reported cstate change: term changed from 0 to 1, leader changed from <none> to b6150724bd8340d1b20cbe1399709b37 (127.5.96.131). New cstate: current_term: 1 leader_uuid: "b6150724bd8340d1b20cbe1399709b37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "197d9e0fbc8c45e1a1cccf0ae99f2b32" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f9d602a2a0e4b0798b007d031da0dd3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 36701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6150724bd8340d1b20cbe1399709b37" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 45225 } health_report { overall_health: HEALTHY } } }
W20260812 03:07:09.210544 2427 tablet.cc:2369] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:07:09.246033 2453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:09.243181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47792 (local address 127.5.96.131:45225)
0812 03:07:09.243284 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:09.243287 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:09.243300 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:07:09.243419 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:09.243421 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:09.243470 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:09.243548 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:07:09.243735 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.244187 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.244933 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.245122 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.245185 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:09.245391 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:07:09.245763 (+ 372us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:07:09.245842 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:09.245889 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 03:07:09.245931 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 03:07:09.251145 2150 raft_consensus.cc:1261] T a0145646b06845229ef620fa11d0e78b P 3f9d602a2a0e4b0798b007d031da0dd3 [term 1 FOLLOWER]: Refusing update from remote peer b6150724bd8340d1b20cbe1399709b37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:07:09.251145 1937 raft_consensus.cc:1261] T a0145646b06845229ef620fa11d0e78b P 197d9e0fbc8c45e1a1cccf0ae99f2b32 [term 1 FOLLOWER]: Refusing update from remote peer b6150724bd8340d1b20cbe1399709b37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:07:09.251729 2463 consensus_queue.cc:1035] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f9d602a2a0e4b0798b007d031da0dd3" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 36701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:07:09.251907 2469 consensus_queue.cc:1035] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "197d9e0fbc8c45e1a1cccf0ae99f2b32" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:07:09.258839 2475 mvcc.cc:205] Tried to move back new op lower bound from 7317520503806476288 to 7317520503627816960. Current Snapshot: MvccSnapshot[applied={T|T < 7317520503806476288}]
I20260812 03:07:09.259691 2473 mvcc.cc:205] Tried to move back new op lower bound from 7317520503806476288 to 7317520503627816960. Current Snapshot: MvccSnapshot[applied={T|T < 7317520503806476288}]
I20260812 03:07:09.266753 1703 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:07:09.266888 1703 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:07:09.268986 1703 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 855aee73f62f4c3fb666c56c9a412525: Sending DeleteTablet for 3 replicas of tablet a0145646b06845229ef620fa11d0e78b
I20260812 03:07:09.269637 2117 tablet_service.cc:1514] Processing DeleteTablet for tablet a0145646b06845229ef620fa11d0e78b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:07:09 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:57122
I20260812 03:07:09.269657 1908 tablet_service.cc:1514] Processing DeleteTablet for tablet a0145646b06845229ef620fa11d0e78b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:07:09 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:45426
I20260812 03:07:09.269743 2321 tablet_service.cc:1514] Processing DeleteTablet for tablet a0145646b06845229ef620fa11d0e78b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:07:09 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:47786
I20260812 03:07:09.270150 2498 tablet_replica.cc:330] T a0145646b06845229ef620fa11d0e78b P 197d9e0fbc8c45e1a1cccf0ae99f2b32: stopping tablet replica
I20260812 03:07:09.270342 2497 tablet_replica.cc:330] T a0145646b06845229ef620fa11d0e78b P 3f9d602a2a0e4b0798b007d031da0dd3: stopping tablet replica
I20260812 03:07:09.270466 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 1805
I20260812 03:07:09.270613 2497 raft_consensus.cc:2229] T a0145646b06845229ef620fa11d0e78b P 3f9d602a2a0e4b0798b007d031da0dd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:07:09.270892 2497 raft_consensus.cc:2258] T a0145646b06845229ef620fa11d0e78b P 3f9d602a2a0e4b0798b007d031da0dd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:07:09.271195 2499 tablet_replica.cc:330] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37: stopping tablet replica
I20260812 03:07:09.271445 2499 raft_consensus.cc:2229] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:07:09.271778 2499 raft_consensus.cc:2258] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:07:09.276151 2497 ts_tablet_manager.cc:1905] T a0145646b06845229ef620fa11d0e78b P 3f9d602a2a0e4b0798b007d031da0dd3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:07:09.279004 2497 ts_tablet_manager.cc:1918] T a0145646b06845229ef620fa11d0e78b P 3f9d602a2a0e4b0798b007d031da0dd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:07:09.279121 2499 ts_tablet_manager.cc:1905] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:07:09.279152 2497 log.cc:1196] T a0145646b06845229ef620fa11d0e78b P 3f9d602a2a0e4b0798b007d031da0dd3: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-1/wal/wals/a0145646b06845229ef620fa11d0e78b
I20260812 03:07:09.279435 2497 ts_tablet_manager.cc:1939] T a0145646b06845229ef620fa11d0e78b P 3f9d602a2a0e4b0798b007d031da0dd3: Deleting consensus metadata
I20260812 03:07:09.280901 1684 catalog_manager.cc:4841] TS 3f9d602a2a0e4b0798b007d031da0dd3 (127.5.96.130:36701): tablet a0145646b06845229ef620fa11d0e78b (table test-table [id=8f1076c422f24e41b0489e725739b755]) successfully deleted
I20260812 03:07:09.281854 2499 ts_tablet_manager.cc:1918] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:07:09.281948 2499 log.cc:1196] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786503790637001-5506-0/minicluster-data/ts-2/wal/wals/a0145646b06845229ef620fa11d0e78b
I20260812 03:07:09.282311 2499 ts_tablet_manager.cc:1939] T a0145646b06845229ef620fa11d0e78b P b6150724bd8340d1b20cbe1399709b37: Deleting consensus metadata
I20260812 03:07:09.283396 1685 catalog_manager.cc:4841] TS b6150724bd8340d1b20cbe1399709b37 (127.5.96.131:45225): tablet a0145646b06845229ef620fa11d0e78b (table test-table [id=8f1076c422f24e41b0489e725739b755]) successfully deleted
W20260812 03:07:09.287634 1686 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.96.129:34235 (error 108)
I20260812 03:07:09.287802 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 2010
I20260812 03:07:09.296614 2447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:09.288448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:34235 (local address 127.0.0.1:45438)
0812 03:07:09.296447 (+ 7999us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:09.296526 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.129:34235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7902}
W20260812 03:07:09.296787 1686 catalog_manager.cc:4584] TS 197d9e0fbc8c45e1a1cccf0ae99f2b32 (127.5.96.129:34235): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a0145646b06845229ef620fa11d0e78b: Network error: Client connection negotiation failed: client connection to 127.5.96.129:34235: connect: Connection refused (error 111)
I20260812 03:07:09.303097 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 2223
I20260812 03:07:09.317364 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 1656
2026-08-12T03:07:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2440 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T03:07:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:09Z Disabled control of system clock
I20260812 03:07:09.349093 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:42833
--webserver_interface=127.5.96.190
--webserver_port=0
--builtin_ntp_servers=127.5.96.148:40503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:42833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:07:09.456902 2504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:09.457125 2504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:09.457157 2504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:09.460989 2504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:09.461035 2504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:09.461048 2504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:09.461063 2504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:09.461082 2504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:09.465561 2504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:42833
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:42833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:09.466410 2504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:09.467679 2504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:09.475562 2512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:09.475538 2511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:09.475586 2504 server_base.cc:1034] running on GCE node
W20260812 03:07:09.475732 2516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:09.476387 2504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:09.477370 2504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:09.478518 2504 hybrid_clock.cc:648] HybridClock initialized: now 1786504029478503 us; error 24 us; skew 500 ppm
I20260812 03:07:09.480242 2504 webserver.cc:460] Webserver started at http://127.5.96.190:36669/ using document root <none> and password file <none>
I20260812 03:07:09.480762 2504 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:09.480824 2504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:09.481017 2504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:09.482904 2504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "b6e42d60a07d486a80bfab4de27df4da"
format_stamp: "Formatted at 2026-08-12 03:07:09 on dist-test-slave-6bbx"
server_key: "6106df83666c132030d539106ba509e3"
server_key_iv: "2157e13f681c55d4f0dd20ba47996569"
server_key_version: "encryptionkey@0"
I20260812 03:07:09.483443 2504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "b6e42d60a07d486a80bfab4de27df4da"
format_stamp: "Formatted at 2026-08-12 03:07:09 on dist-test-slave-6bbx"
server_key: "6106df83666c132030d539106ba509e3"
server_key_iv: "2157e13f681c55d4f0dd20ba47996569"
server_key_version: "encryptionkey@0"
I20260812 03:07:09.486768 2504 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:07:09.488983 2529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:09.490059 2504 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:07:09.490161 2504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "b6e42d60a07d486a80bfab4de27df4da"
format_stamp: "Formatted at 2026-08-12 03:07:09 on dist-test-slave-6bbx"
server_key: "6106df83666c132030d539106ba509e3"
server_key_iv: "2157e13f681c55d4f0dd20ba47996569"
server_key_version: "encryptionkey@0"
I20260812 03:07:09.490258 2504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:09.501591 2504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:09.502164 2504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:09.502331 2504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:09.508632 2504 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:42833
I20260812 03:07:09.508646 2605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:42833 every 8 connection(s)
I20260812 03:07:09.509475 2504 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:09.512558 2606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:09.513934 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 2504
I20260812 03:07:09.514068 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:07:09.514387 5506 external_mini_cluster.cc:1428] Setting key 4b2cf5a94c46390a1aff133a418f23c9
I20260812 03:07:09.518601 2606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6e42d60a07d486a80bfab4de27df4da: Bootstrap starting.
I20260812 03:07:09.520716 2606 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6e42d60a07d486a80bfab4de27df4da: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:09.521404 2606 log.cc:824] T 00000000000000000000000000000000 P b6e42d60a07d486a80bfab4de27df4da: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:09.523147 2614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:09.515850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36118 (local address 127.5.96.190:42833)
0812 03:07:09.516453 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:09.516468 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:09.516502 (+ 34us) server_negotiation.cc:407] Connection header received
0812 03:07:09.517132 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:09.517148 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:09.517519 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:09.517822 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:09.518218 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.519165 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.520066 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.520508 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.520649 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:09.521406 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:09.521411 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:09.521423 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:09.521860 (+ 437us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:09.521929 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:09.521957 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:09.522053 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:09.522235 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 03:07:09.522486 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":124,"threads_started":1}
I20260812 03:07:09.523272 2606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6e42d60a07d486a80bfab4de27df4da: No bootstrap required, opened a new log
I20260812 03:07:09.526934 2606 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6e42d60a07d486a80bfab4de27df4da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6e42d60a07d486a80bfab4de27df4da" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42833 } }
I20260812 03:07:09.527137 2606 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6e42d60a07d486a80bfab4de27df4da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:09.527192 2606 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6e42d60a07d486a80bfab4de27df4da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6e42d60a07d486a80bfab4de27df4da, State: Initialized, Role: FOLLOWER
I20260812 03:07:09.527570 2606 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6e42d60a07d486a80bfab4de27df4da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6e42d60a07d486a80bfab4de27df4da" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42833 } }
I20260812 03:07:09.527720 2606 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6e42d60a07d486a80bfab4de27df4da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:07:09.527783 2606 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6e42d60a07d486a80bfab4de27df4da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:07:09.527880 2606 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6e42d60a07d486a80bfab4de27df4da [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:09.528656 2606 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6e42d60a07d486a80bfab4de27df4da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6e42d60a07d486a80bfab4de27df4da" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42833 } }
I20260812 03:07:09.528962 2606 leader_election.cc:302] T 00000000000000000000000000000000 P b6e42d60a07d486a80bfab4de27df4da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6e42d60a07d486a80bfab4de27df4da; no voters:
I20260812 03:07:09.529227 2606 leader_election.cc:288] T 00000000000000000000000000000000 P b6e42d60a07d486a80bfab4de27df4da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:07:09.529347 2616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6e42d60a07d486a80bfab4de27df4da [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:09.529615 2616 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6e42d60a07d486a80bfab4de27df4da [term 1 LEADER]: Becoming Leader. State: Replica: b6e42d60a07d486a80bfab4de27df4da, State: Running, Role: LEADER
I20260812 03:07:09.529938 2616 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6e42d60a07d486a80bfab4de27df4da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6e42d60a07d486a80bfab4de27df4da" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42833 } }
I20260812 03:07:09.530380 2606 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6e42d60a07d486a80bfab4de27df4da [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:09.531416 2618 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e42d60a07d486a80bfab4de27df4da [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6e42d60a07d486a80bfab4de27df4da. Latest consensus state: current_term: 1 leader_uuid: "b6e42d60a07d486a80bfab4de27df4da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6e42d60a07d486a80bfab4de27df4da" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42833 } } }
I20260812 03:07:09.531543 2618 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e42d60a07d486a80bfab4de27df4da [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:09.531849 2617 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e42d60a07d486a80bfab4de27df4da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6e42d60a07d486a80bfab4de27df4da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6e42d60a07d486a80bfab4de27df4da" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42833 } } }
I20260812 03:07:09.531948 2617 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e42d60a07d486a80bfab4de27df4da [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:09.531913 2631 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:09.534791 2631 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:09.539594 2631 catalog_manager.cc:1347] Generated new cluster ID: ed4d04bec83d471eac522e47013e44b3
I20260812 03:07:09.539654 2631 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:09.549085 2631 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:09.550189 2631 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:09.556062 2631 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6e42d60a07d486a80bfab4de27df4da: Generated new TSK 0
I20260812 03:07:09.556748 2631 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:07:09.559085 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:0
--local_ip_for_outbound_sockets=127.5.96.129
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:42833
--builtin_ntp_servers=127.5.96.148:40503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:07:09.665912 2645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:09.666136 2645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:09.666185 2645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:09.670050 2645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:09.670095 2645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:09.670166 2645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:07:09.674680 2645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:42833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:09.675572 2645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:09.676841 2645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:09.685941 2658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:09.686007 2657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:09.686010 2662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:09.686226 2645 server_base.cc:1034] running on GCE node
I20260812 03:07:09.686594 2645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:09.687179 2645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:09.688338 2645 hybrid_clock.cc:648] HybridClock initialized: now 1786504029688322 us; error 29 us; skew 500 ppm
I20260812 03:07:09.689993 2645 webserver.cc:460] Webserver started at http://127.5.96.129:37593/ using document root <none> and password file <none>
I20260812 03:07:09.690500 2645 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:09.690564 2645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:09.690752 2645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:09.692658 2645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-0/data/instance:
uuid: "39975d450c614610be98f278819f7ee7"
format_stamp: "Formatted at 2026-08-12 03:07:09 on dist-test-slave-6bbx"
server_key: "9727f922250874382eb82e2b134a16c5"
server_key_iv: "08b0ed5459c9bbc6c18ce83619a34dd3"
server_key_version: "encryptionkey@0"
I20260812 03:07:09.693202 2645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance:
uuid: "39975d450c614610be98f278819f7ee7"
format_stamp: "Formatted at 2026-08-12 03:07:09 on dist-test-slave-6bbx"
server_key: "9727f922250874382eb82e2b134a16c5"
server_key_iv: "08b0ed5459c9bbc6c18ce83619a34dd3"
server_key_version: "encryptionkey@0"
I20260812 03:07:09.696561 2645 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 03:07:09.698724 2674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:09.699877 2645 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260812 03:07:09.699980 2645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "39975d450c614610be98f278819f7ee7"
format_stamp: "Formatted at 2026-08-12 03:07:09 on dist-test-slave-6bbx"
server_key: "9727f922250874382eb82e2b134a16c5"
server_key_iv: "08b0ed5459c9bbc6c18ce83619a34dd3"
server_key_version: "encryptionkey@0"
I20260812 03:07:09.700067 2645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:09.736286 2645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:09.736881 2645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:09.737041 2645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:09.737565 2645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:07:09.738451 2645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:07:09.738499 2645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:09.738549 2645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:07:09.738579 2645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:09.746529 2645 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:37421
I20260812 03:07:09.746587 2850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:37421 every 8 connection(s)
I20260812 03:07:09.747381 2645 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:07:09.755599 2614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:09.749023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:33705 (local address 127.5.96.190:42833)
0812 03:07:09.749150 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:09.749154 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:09.750081 (+ 927us) server_negotiation.cc:407] Connection header received
0812 03:07:09.751348 (+ 1267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:09.751350 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:09.751394 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:09.751461 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:09.752277 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.752809 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.753896 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.754123 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.754214 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:09.755135 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:09.755138 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:09.755140 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:09.755151 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:09.755169 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:09.755177 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:09.755236 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:09.755385 (+ 149us) server_negotiation.cc:299] Negotiation successful
0812 03:07:09.755449 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:07:09.755918 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 2645
I20260812 03:07:09.756017 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance
I20260812 03:07:09.756282 5506 external_mini_cluster.cc:1428] Setting key bd0dd3080f225e120492040139603cef
I20260812 03:07:09.756456 2853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:09.749370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:42833 (local address 127.5.96.129:33705)
0812 03:07:09.749967 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:09.750004 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:09.751173 (+ 1169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:09.751597 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:09.751603 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:09.751719 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:09.752170 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:09.752182 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.752893 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.752896 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:09.753771 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:09.753781 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.754268 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.754273 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:09.754423 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:09.754899 (+ 476us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:07:09.754900 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:09.754936 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:07:09.754994 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:09.755245 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:09.755265 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:09.755271 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:07:09.755372 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 03:07:09.755623 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":113,"threads_started":1}
I20260812 03:07:09.757493 2851 heartbeater.cc:347] Connected to a master server at 127.5.96.190:42833
I20260812 03:07:09.757700 2851 heartbeater.cc:464] Registering TS with master...
I20260812 03:07:09.758257 2851 heartbeater.cc:511] Master 127.5.96.190:42833 requested a full tablet report, sending...
I20260812 03:07:09.758322 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:0
--local_ip_for_outbound_sockets=127.5.96.130
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:42833
--builtin_ntp_servers=127.5.96.148:40503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 03:07:09.759635 2555 ts_manager.cc:194] Registered new tserver with Master: 39975d450c614610be98f278819f7ee7 (127.5.96.129:37421)
I20260812 03:07:09.761111 2555 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.96.129:33705
W20260812 03:07:09.867772 2854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:09.867997 2854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:09.868032 2854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:09.871950 2854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:09.872002 2854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:09.872054 2854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:07:09.877161 2854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:42833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:09.878033 2854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:09.879246 2854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:09.888267 2866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:09.888279 2869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:09.888342 2854 server_base.cc:1034] running on GCE node
W20260812 03:07:09.888274 2871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:09.888792 2854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:09.889367 2854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:09.890524 2854 hybrid_clock.cc:648] HybridClock initialized: now 1786504029890508 us; error 26 us; skew 500 ppm
I20260812 03:07:09.892181 2854 webserver.cc:460] Webserver started at http://127.5.96.130:40261/ using document root <none> and password file <none>
I20260812 03:07:09.892709 2854 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:09.892776 2854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:09.892956 2854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:09.894933 2854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-1/data/instance:
uuid: "92f8e0876fed4e35a81e6aa5720a8e0b"
format_stamp: "Formatted at 2026-08-12 03:07:09 on dist-test-slave-6bbx"
server_key: "6f44d6062dccfda3b05f811e74d9a5ff"
server_key_iv: "af6d0800b3034917ab979cae698e2a11"
server_key_version: "encryptionkey@0"
I20260812 03:07:09.895453 2854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance:
uuid: "92f8e0876fed4e35a81e6aa5720a8e0b"
format_stamp: "Formatted at 2026-08-12 03:07:09 on dist-test-slave-6bbx"
server_key: "6f44d6062dccfda3b05f811e74d9a5ff"
server_key_iv: "af6d0800b3034917ab979cae698e2a11"
server_key_version: "encryptionkey@0"
I20260812 03:07:09.898823 2854 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:07:09.901139 2884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:09.902206 2854 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:07:09.902319 2854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "92f8e0876fed4e35a81e6aa5720a8e0b"
format_stamp: "Formatted at 2026-08-12 03:07:09 on dist-test-slave-6bbx"
server_key: "6f44d6062dccfda3b05f811e74d9a5ff"
server_key_iv: "af6d0800b3034917ab979cae698e2a11"
server_key_version: "encryptionkey@0"
I20260812 03:07:09.902406 2854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:09.923894 2854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:09.924453 2854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:09.924600 2854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:09.925099 2854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:07:09.925941 2854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:07:09.925989 2854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:09.926038 2854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:07:09.926064 2854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:09.934078 2854 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:37357
I20260812 03:07:09.934113 3072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:37357 every 8 connection(s)
I20260812 03:07:09.934847 2854 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:07:09.935036 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 2854
I20260812 03:07:09.935144 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance
I20260812 03:07:09.935456 5506 external_mini_cluster.cc:1428] Setting key 456efc2c07e6d7899a75ab345ef38fd5
I20260812 03:07:09.937561 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:0
--local_ip_for_outbound_sockets=127.5.96.131
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:42833
--builtin_ntp_servers=127.5.96.148:40503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 03:07:09.943585 2614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:09.936552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:56345 (local address 127.5.96.190:42833)
0812 03:07:09.936690 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:09.936695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:09.937787 (+ 1092us) server_negotiation.cc:407] Connection header received
0812 03:07:09.939165 (+ 1378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:09.939168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:09.939221 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:09.939301 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:09.940210 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.940726 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.941836 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.942061 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.942163 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:09.943131 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:09.943135 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:09.943137 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:09.943150 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:09.943170 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:09.943178 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:09.943236 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:09.943375 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 03:07:09.943437 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:07:09.944422 3075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:09.936991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:42833 (local address 127.5.96.130:56345)
0812 03:07:09.937632 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:09.937683 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:09.938952 (+ 1269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:09.939462 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:09.939468 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:09.939549 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:09.940079 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:09.940092 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.940840 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.940843 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:09.941711 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:09.941721 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:09.942206 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:09.942210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:09.942365 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:09.942890 (+ 525us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:07:09.942892 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:09.942930 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:07:09.942991 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:09.943249 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:09.943258 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:09.943264 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:07:09.943366 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 03:07:09.943623 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":127,"threads_started":1}
I20260812 03:07:09.945425 3073 heartbeater.cc:347] Connected to a master server at 127.5.96.190:42833
I20260812 03:07:09.945632 3073 heartbeater.cc:464] Registering TS with master...
I20260812 03:07:09.946084 3073 heartbeater.cc:511] Master 127.5.96.190:42833 requested a full tablet report, sending...
I20260812 03:07:09.947057 2555 ts_manager.cc:194] Registered new tserver with Master: 92f8e0876fed4e35a81e6aa5720a8e0b (127.5.96.130:37357)
I20260812 03:07:09.947835 2555 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.96.130:56345
W20260812 03:07:10.046545 3079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:10.046756 3079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:10.046790 3079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:10.050607 3079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:10.050650 3079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:10.050699 3079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:07:10.055150 3079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:40503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:42833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:10.056052 3079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:10.057236 3079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:10.065924 3087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:10.065927 3089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:10.065943 3091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:10.066175 3079 server_base.cc:1034] running on GCE node
I20260812 03:07:10.066510 3079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:10.067296 3079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:10.068457 3079 hybrid_clock.cc:648] HybridClock initialized: now 1786504030068453 us; error 47 us; skew 500 ppm
I20260812 03:07:10.070106 3079 webserver.cc:460] Webserver started at http://127.5.96.131:38313/ using document root <none> and password file <none>
I20260812 03:07:10.070608 3079 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:10.070668 3079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:10.070859 3079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:10.072728 3079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-2/data/instance:
uuid: "ce8da99cd776432a9dd696240ec33e3a"
format_stamp: "Formatted at 2026-08-12 03:07:10 on dist-test-slave-6bbx"
server_key: "267b9b2557abc6ca1aca2431e20b8d55"
server_key_iv: "a0fa3af4df2c0feccffcb023037d7aea"
server_key_version: "encryptionkey@0"
I20260812 03:07:10.073232 3079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance:
uuid: "ce8da99cd776432a9dd696240ec33e3a"
format_stamp: "Formatted at 2026-08-12 03:07:10 on dist-test-slave-6bbx"
server_key: "267b9b2557abc6ca1aca2431e20b8d55"
server_key_iv: "a0fa3af4df2c0feccffcb023037d7aea"
server_key_version: "encryptionkey@0"
I20260812 03:07:10.076555 3079 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:07:10.078646 3102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:10.079764 3079 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:07:10.079867 3079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "ce8da99cd776432a9dd696240ec33e3a"
format_stamp: "Formatted at 2026-08-12 03:07:10 on dist-test-slave-6bbx"
server_key: "267b9b2557abc6ca1aca2431e20b8d55"
server_key_iv: "a0fa3af4df2c0feccffcb023037d7aea"
server_key_version: "encryptionkey@0"
I20260812 03:07:10.079957 3079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:10.091784 3079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:10.092314 3079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:10.092473 3079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:10.092939 3079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:07:10.093779 3079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:07:10.093825 3079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:10.093859 3079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:07:10.093879 3079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:10.101671 3079 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:44749
I20260812 03:07:10.101728 3269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:44749 every 8 connection(s)
I20260812 03:07:10.102458 3079 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:07:10.104030 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 3079
I20260812 03:07:10.104135 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance
I20260812 03:07:10.104425 5506 external_mini_cluster.cc:1428] Setting key 0c51b10f7d81ece030e00e1bc821a77f
I20260812 03:07:10.111315 2614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:10.104116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:37387 (local address 127.5.96.190:42833)
0812 03:07:10.104242 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:10.104257 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:10.105490 (+ 1233us) server_negotiation.cc:407] Connection header received
0812 03:07:10.107038 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:10.107041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:10.107095 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:10.107164 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:10.108011 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.108525 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.109607 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.109835 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.109921 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:10.110867 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:10.110871 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:10.110873 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:10.110887 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:10.110906 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:10.110913 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:10.110971 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:10.111113 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 03:07:10.111175 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 03:07:10.112054 3276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:10.104616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:42833 (local address 127.5.96.131:37387)
0812 03:07:10.105350 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:10.105402 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:10.106847 (+ 1445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:10.107341 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:10.107348 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:10.107424 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:10.107881 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:10.107894 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.108638 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.108642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:10.109487 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:10.109496 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.109962 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.109966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:10.110119 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:10.110633 (+ 514us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:07:10.110634 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:10.110671 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:07:10.110727 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:10.110982 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:10.110991 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:10.110997 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:07:10.111121 (+ 124us) client_negotiation.cc:240] Negotiation successful
0812 03:07:10.111377 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":121,"threads_started":1}
I20260812 03:07:10.112968 3272 heartbeater.cc:347] Connected to a master server at 127.5.96.190:42833
I20260812 03:07:10.113159 3272 heartbeater.cc:464] Registering TS with master...
I20260812 03:07:10.113606 3272 heartbeater.cc:511] Master 127.5.96.190:42833 requested a full tablet report, sending...
I20260812 03:07:10.114480 2555 ts_manager.cc:194] Registered new tserver with Master: ce8da99cd776432a9dd696240ec33e3a (127.5.96.131:44749)
I20260812 03:07:10.115000 2555 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.96.131:37387
I20260812 03:07:10.117897 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:07:10.123072 2614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:10.119641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36126 (local address 127.5.96.190:42833)
0812 03:07:10.119786 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:10.119790 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:10.119937 (+ 147us) server_negotiation.cc:407] Connection header received
0812 03:07:10.120045 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:10.120048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:10.120086 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:10.120146 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:10.120420 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.120897 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.121708 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.121923 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.122004 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:10.122708 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:10.122712 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:10.122715 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:10.122726 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:10.122744 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:10.122759 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:10.122815 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:10.122903 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 03:07:10.122951 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 03:07:10.125392 2555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36126:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:07:10.128260 2555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:07:10.141901 3296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:10.136793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47256 (local address 127.5.96.131:44749)
0812 03:07:10.137109 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:10.137120 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:10.137406 (+ 286us) server_negotiation.cc:407] Connection header received
0812 03:07:10.137483 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:10.137488 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:10.137669 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:10.137815 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:10.138154 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.139266 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.140602 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.141190 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.141288 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:10.141474 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:10.141634 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 03:07:10.141735 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":98,"threads_started":1}
I20260812 03:07:10.141899 3291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:10.136015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:37421 (local address 127.0.0.1:60202)
0812 03:07:10.136537 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:10.136580 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:10.136679 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:10.137791 (+ 1112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:10.137796 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:10.137817 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:10.137931 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:10.137937 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.139362 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.139366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:10.140528 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:10.140536 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.141487 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.141491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:10.141562 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:10.141580 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 03:07:10.141620 (+ 40us) client_negotiation.cc:240] Negotiation successful
0812 03:07:10.141720 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":79,"thread_start_us":73,"threads_started":1}
I20260812 03:07:10.141901 3295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:10.136178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:37357 (local address 127.0.0.1:59874)
0812 03:07:10.136893 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:10.136906 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:10.137346 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:10.137983 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:10.137986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:10.138006 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:10.138095 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:10.138101 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.139022 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.139025 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:10.140349 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:10.140360 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.141395 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.141399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:10.141514 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:10.141540 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 03:07:10.141585 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 03:07:10.141735 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"mutex_wait_us":26,"thread_start_us":132,"threads_started":1}
I20260812 03:07:10.142190 3293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:10.136156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60202 (local address 127.5.96.129:37421)
0812 03:07:10.137322 (+ 1166us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:10.137333 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:10.137358 (+ 25us) server_negotiation.cc:407] Connection header received
0812 03:07:10.137437 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:10.137442 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:10.137640 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:10.137775 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:10.138054 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.139245 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.140634 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.141379 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.141697 (+ 318us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:10.141878 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:10.141928 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 03:07:10.142021 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1054,"thread_start_us":143,"threads_started":1}
I20260812 03:07:10.142352 3292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:10.136643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:44749 (local address 127.0.0.1:47256)
0812 03:07:10.137205 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:10.137217 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:10.137312 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:10.137936 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:10.137939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:10.137954 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:10.138036 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:10.138042 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.139385 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.139388 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:10.140483 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:10.140493 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.141307 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.141311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:10.141450 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:10.141491 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 03:07:10.141590 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 03:07:10.142223 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"mutex_wait_us":66,"thread_start_us":108,"threads_started":1}
I20260812 03:07:10.142707 3297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:10.137019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59874 (local address 127.5.96.130:37357)
0812 03:07:10.137573 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:10.137585 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:10.137604 (+ 19us) server_negotiation.cc:407] Connection header received
0812 03:07:10.137669 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:10.137673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:10.137829 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:10.137999 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:10.138183 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.138912 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.140483 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.141252 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.142243 (+ 991us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:10.142416 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:10.142465 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 03:07:10.142547 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":99,"threads_started":1}
I20260812 03:07:10.145152 3171 tablet_service.cc:1467] Processing CreateTablet for tablet 6ef7ad9ef0e64dc6a802a510327d5142 (DEFAULT_TABLE table=test-table [id=1e6aa1aed22e439095d4fb0d7d103501]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:07:10.145192 2754 tablet_service.cc:1467] Processing CreateTablet for tablet 6ef7ad9ef0e64dc6a802a510327d5142 (DEFAULT_TABLE table=test-table [id=1e6aa1aed22e439095d4fb0d7d103501]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:07:10.145435 2975 tablet_service.cc:1467] Processing CreateTablet for tablet 6ef7ad9ef0e64dc6a802a510327d5142 (DEFAULT_TABLE table=test-table [id=1e6aa1aed22e439095d4fb0d7d103501]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:07:10.146103 3171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ef7ad9ef0e64dc6a802a510327d5142. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:10.146211 2754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ef7ad9ef0e64dc6a802a510327d5142. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:10.146220 2975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ef7ad9ef0e64dc6a802a510327d5142. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:10.152204 3298 tablet_bootstrap.cc:492] T 6ef7ad9ef0e64dc6a802a510327d5142 P ce8da99cd776432a9dd696240ec33e3a: Bootstrap starting.
I20260812 03:07:10.153159 3302 tablet_bootstrap.cc:492] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b: Bootstrap starting.
I20260812 03:07:10.153772 3300 tablet_bootstrap.cc:492] T 6ef7ad9ef0e64dc6a802a510327d5142 P 39975d450c614610be98f278819f7ee7: Bootstrap starting.
I20260812 03:07:10.154506 3298 tablet_bootstrap.cc:654] T 6ef7ad9ef0e64dc6a802a510327d5142 P ce8da99cd776432a9dd696240ec33e3a: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:10.155186 3302 tablet_bootstrap.cc:654] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:10.155351 3298 log.cc:824] T 6ef7ad9ef0e64dc6a802a510327d5142 P ce8da99cd776432a9dd696240ec33e3a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:10.155405 3300 tablet_bootstrap.cc:654] T 6ef7ad9ef0e64dc6a802a510327d5142 P 39975d450c614610be98f278819f7ee7: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:10.155956 3302 log.cc:824] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:10.156165 3300 log.cc:824] T 6ef7ad9ef0e64dc6a802a510327d5142 P 39975d450c614610be98f278819f7ee7: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:10.157352 3298 tablet_bootstrap.cc:492] T 6ef7ad9ef0e64dc6a802a510327d5142 P ce8da99cd776432a9dd696240ec33e3a: No bootstrap required, opened a new log
I20260812 03:07:10.157543 3298 ts_tablet_manager.cc:1397] T 6ef7ad9ef0e64dc6a802a510327d5142 P ce8da99cd776432a9dd696240ec33e3a: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260812 03:07:10.157821 3302 tablet_bootstrap.cc:492] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b: No bootstrap required, opened a new log
I20260812 03:07:10.157974 3300 tablet_bootstrap.cc:492] T 6ef7ad9ef0e64dc6a802a510327d5142 P 39975d450c614610be98f278819f7ee7: No bootstrap required, opened a new log
I20260812 03:07:10.158006 3302 ts_tablet_manager.cc:1397] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 03:07:10.158143 3300 ts_tablet_manager.cc:1397] T 6ef7ad9ef0e64dc6a802a510327d5142 P 39975d450c614610be98f278819f7ee7: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260812 03:07:10.161064 3300 raft_consensus.cc:348] T 6ef7ad9ef0e64dc6a802a510327d5142 P 39975d450c614610be98f278819f7ee7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39975d450c614610be98f278819f7ee7" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 37421 } } peers { permanent_uuid: "92f8e0876fed4e35a81e6aa5720a8e0b" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 37357 } } peers { permanent_uuid: "ce8da99cd776432a9dd696240ec33e3a" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44749 } }
I20260812 03:07:10.161279 3300 raft_consensus.cc:374] T 6ef7ad9ef0e64dc6a802a510327d5142 P 39975d450c614610be98f278819f7ee7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:10.161334 3300 raft_consensus.cc:729] T 6ef7ad9ef0e64dc6a802a510327d5142 P 39975d450c614610be98f278819f7ee7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39975d450c614610be98f278819f7ee7, State: Initialized, Role: FOLLOWER
I20260812 03:07:10.161716 3300 consensus_queue.cc:260] T 6ef7ad9ef0e64dc6a802a510327d5142 P 39975d450c614610be98f278819f7ee7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39975d450c614610be98f278819f7ee7" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 37421 } } peers { permanent_uuid: "92f8e0876fed4e35a81e6aa5720a8e0b" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 37357 } } peers { permanent_uuid: "ce8da99cd776432a9dd696240ec33e3a" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44749 } }
I20260812 03:07:10.161664 3298 raft_consensus.cc:348] T 6ef7ad9ef0e64dc6a802a510327d5142 P ce8da99cd776432a9dd696240ec33e3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39975d450c614610be98f278819f7ee7" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 37421 } } peers { permanent_uuid: "92f8e0876fed4e35a81e6aa5720a8e0b" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 37357 } } peers { permanent_uuid: "ce8da99cd776432a9dd696240ec33e3a" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44749 } }
I20260812 03:07:10.161906 3298 raft_consensus.cc:374] T 6ef7ad9ef0e64dc6a802a510327d5142 P ce8da99cd776432a9dd696240ec33e3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:10.161958 3298 raft_consensus.cc:729] T 6ef7ad9ef0e64dc6a802a510327d5142 P ce8da99cd776432a9dd696240ec33e3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce8da99cd776432a9dd696240ec33e3a, State: Initialized, Role: FOLLOWER
I20260812 03:07:10.161849 3302 raft_consensus.cc:348] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39975d450c614610be98f278819f7ee7" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 37421 } } peers { permanent_uuid: "92f8e0876fed4e35a81e6aa5720a8e0b" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 37357 } } peers { permanent_uuid: "ce8da99cd776432a9dd696240ec33e3a" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44749 } }
I20260812 03:07:10.162070 3302 raft_consensus.cc:374] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:10.162144 3302 raft_consensus.cc:729] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92f8e0876fed4e35a81e6aa5720a8e0b, State: Initialized, Role: FOLLOWER
I20260812 03:07:10.162392 3298 consensus_queue.cc:260] T 6ef7ad9ef0e64dc6a802a510327d5142 P ce8da99cd776432a9dd696240ec33e3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39975d450c614610be98f278819f7ee7" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 37421 } } peers { permanent_uuid: "92f8e0876fed4e35a81e6aa5720a8e0b" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 37357 } } peers { permanent_uuid: "ce8da99cd776432a9dd696240ec33e3a" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44749 } }
I20260812 03:07:10.162537 2851 heartbeater.cc:503] Master 127.5.96.190:42833 was elected leader, sending a full tablet report...
I20260812 03:07:10.162658 3300 ts_tablet_manager.cc:1428] T 6ef7ad9ef0e64dc6a802a510327d5142 P 39975d450c614610be98f278819f7ee7: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260812 03:07:10.162598 3302 consensus_queue.cc:260] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39975d450c614610be98f278819f7ee7" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 37421 } } peers { permanent_uuid: "92f8e0876fed4e35a81e6aa5720a8e0b" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 37357 } } peers { permanent_uuid: "ce8da99cd776432a9dd696240ec33e3a" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44749 } }
I20260812 03:07:10.163309 3298 ts_tablet_manager.cc:1428] T 6ef7ad9ef0e64dc6a802a510327d5142 P ce8da99cd776432a9dd696240ec33e3a: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260812 03:07:10.163394 3302 ts_tablet_manager.cc:1428] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 03:07:10.163493 3073 heartbeater.cc:503] Master 127.5.96.190:42833 was elected leader, sending a full tablet report...
I20260812 03:07:10.163607 3272 heartbeater.cc:503] Master 127.5.96.190:42833 was elected leader, sending a full tablet report...
W20260812 03:07:10.185765 3074 tablet.cc:2369] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:07:10.233933 3314 raft_consensus.cc:482] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:07:10.234118 3314 raft_consensus.cc:504] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39975d450c614610be98f278819f7ee7" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 37421 } } peers { permanent_uuid: "92f8e0876fed4e35a81e6aa5720a8e0b" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 37357 } } peers { permanent_uuid: "ce8da99cd776432a9dd696240ec33e3a" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44749 } }
I20260812 03:07:10.235184 3314 leader_election.cc:288] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39975d450c614610be98f278819f7ee7 (127.5.96.129:37421), ce8da99cd776432a9dd696240ec33e3a (127.5.96.131:44749)
I20260812 03:07:10.238721 3296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:10.235450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:47631 (local address 127.5.96.131:44749)
0812 03:07:10.235557 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:10.235562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:10.235630 (+ 68us) server_negotiation.cc:407] Connection header received
0812 03:07:10.235790 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:10.235793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:10.235845 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:10.235916 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:10.236183 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.236735 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.237789 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.238178 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.238273 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:10.238322 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:10.238537 (+ 215us) server_negotiation.cc:299] Negotiation successful
0812 03:07:10.238602 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:07:10.238739 3293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:10.235450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:34707 (local address 127.5.96.129:37421)
0812 03:07:10.235557 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:10.235562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:10.235873 (+ 311us) server_negotiation.cc:407] Connection header received
0812 03:07:10.235977 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:10.235979 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:10.236027 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:10.236103 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:10.236364 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.236847 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.237815 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.238196 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.238314 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:10.238363 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:10.238565 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 03:07:10.238639 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:07:10.238725 3075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:10.235381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:44749 (local address 127.5.96.130:47631)
0812 03:07:10.235558 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:10.235573 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:10.235689 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:10.235931 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:10.235935 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:10.235954 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:10.236074 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:10.236083 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.236862 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.236865 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:10.237653 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:10.237662 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.238296 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.238299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:10.238395 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:10.238442 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 03:07:10.238499 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 03:07:10.238562 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":4}
I20260812 03:07:10.238745 3315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:10.235381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:37421 (local address 127.5.96.130:34707)
0812 03:07:10.235780 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:10.235793 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:10.235890 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:10.236117 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:10.236119 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:10.236134 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:10.236260 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:10.236267 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.236948 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.236950 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:10.237693 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:10.237701 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.238312 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.238315 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:10.238452 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:10.238469 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 03:07:10.238541 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 03:07:10.238602 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":71,"thread_start_us":130,"threads_started":1}
I20260812 03:07:10.239391 3204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ef7ad9ef0e64dc6a802a510327d5142" candidate_uuid: "92f8e0876fed4e35a81e6aa5720a8e0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce8da99cd776432a9dd696240ec33e3a" is_pre_election: true
I20260812 03:07:10.239516 2785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ef7ad9ef0e64dc6a802a510327d5142" candidate_uuid: "92f8e0876fed4e35a81e6aa5720a8e0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39975d450c614610be98f278819f7ee7" is_pre_election: true
I20260812 03:07:10.239705 3204 raft_consensus.cc:2454] T 6ef7ad9ef0e64dc6a802a510327d5142 P ce8da99cd776432a9dd696240ec33e3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92f8e0876fed4e35a81e6aa5720a8e0b in term 0.
I20260812 03:07:10.239778 2785 raft_consensus.cc:2454] T 6ef7ad9ef0e64dc6a802a510327d5142 P 39975d450c614610be98f278819f7ee7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92f8e0876fed4e35a81e6aa5720a8e0b in term 0.
I20260812 03:07:10.240190 2888 leader_election.cc:302] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975d450c614610be98f278819f7ee7, 92f8e0876fed4e35a81e6aa5720a8e0b; no voters:
I20260812 03:07:10.240424 3314 raft_consensus.cc:2781] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:07:10.240499 3314 raft_consensus.cc:482] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:07:10.240545 3314 raft_consensus.cc:3037] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:10.241504 3314 raft_consensus.cc:504] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39975d450c614610be98f278819f7ee7" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 37421 } } peers { permanent_uuid: "92f8e0876fed4e35a81e6aa5720a8e0b" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 37357 } } peers { permanent_uuid: "ce8da99cd776432a9dd696240ec33e3a" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44749 } }
I20260812 03:07:10.241804 3314 leader_election.cc:288] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b [CANDIDATE]: Term 1 election: Requested vote from peers 39975d450c614610be98f278819f7ee7 (127.5.96.129:37421), ce8da99cd776432a9dd696240ec33e3a (127.5.96.131:44749)
I20260812 03:07:10.242125 2785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ef7ad9ef0e64dc6a802a510327d5142" candidate_uuid: "92f8e0876fed4e35a81e6aa5720a8e0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39975d450c614610be98f278819f7ee7"
I20260812 03:07:10.242139 3204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ef7ad9ef0e64dc6a802a510327d5142" candidate_uuid: "92f8e0876fed4e35a81e6aa5720a8e0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce8da99cd776432a9dd696240ec33e3a"
I20260812 03:07:10.242225 2785 raft_consensus.cc:3037] T 6ef7ad9ef0e64dc6a802a510327d5142 P 39975d450c614610be98f278819f7ee7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:10.242229 3204 raft_consensus.cc:3037] T 6ef7ad9ef0e64dc6a802a510327d5142 P ce8da99cd776432a9dd696240ec33e3a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:10.243148 3204 raft_consensus.cc:2454] T 6ef7ad9ef0e64dc6a802a510327d5142 P ce8da99cd776432a9dd696240ec33e3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92f8e0876fed4e35a81e6aa5720a8e0b in term 1.
I20260812 03:07:10.243206 2785 raft_consensus.cc:2454] T 6ef7ad9ef0e64dc6a802a510327d5142 P 39975d450c614610be98f278819f7ee7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92f8e0876fed4e35a81e6aa5720a8e0b in term 1.
I20260812 03:07:10.243428 2890 leader_election.cc:302] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92f8e0876fed4e35a81e6aa5720a8e0b, ce8da99cd776432a9dd696240ec33e3a; no voters:
I20260812 03:07:10.243580 3314 raft_consensus.cc:2781] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:10.243794 3314 raft_consensus.cc:686] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b [term 1 LEADER]: Becoming Leader. State: Replica: 92f8e0876fed4e35a81e6aa5720a8e0b, State: Running, Role: LEADER
I20260812 03:07:10.244069 3314 consensus_queue.cc:237] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39975d450c614610be98f278819f7ee7" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 37421 } } peers { permanent_uuid: "92f8e0876fed4e35a81e6aa5720a8e0b" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 37357 } } peers { permanent_uuid: "ce8da99cd776432a9dd696240ec33e3a" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44749 } }
I20260812 03:07:10.246927 2555 catalog_manager.cc:5495] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 92f8e0876fed4e35a81e6aa5720a8e0b (127.5.96.130). New cstate: current_term: 1 leader_uuid: "92f8e0876fed4e35a81e6aa5720a8e0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39975d450c614610be98f278819f7ee7" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 37421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92f8e0876fed4e35a81e6aa5720a8e0b" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 37357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce8da99cd776432a9dd696240ec33e3a" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44749 } health_report { overall_health: UNKNOWN } } }
W20260812 03:07:10.248443 2852 tablet.cc:2369] T 6ef7ad9ef0e64dc6a802a510327d5142 P 39975d450c614610be98f278819f7ee7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:07:10.260198 3297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:10.257378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59878 (local address 127.5.96.130:37357)
0812 03:07:10.257471 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:10.257474 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:10.257485 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:07:10.257593 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:10.257595 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:10.257640 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:10.257716 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:07:10.257890 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.258343 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.259078 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:10.259274 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:10.259336 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:10.259535 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:07:10.259921 (+ 386us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:07:10.260001 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:10.260054 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 03:07:10.260093 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 03:07:10.265093 2785 raft_consensus.cc:1261] T 6ef7ad9ef0e64dc6a802a510327d5142 P 39975d450c614610be98f278819f7ee7 [term 1 FOLLOWER]: Refusing update from remote peer 92f8e0876fed4e35a81e6aa5720a8e0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:07:10.265094 3204 raft_consensus.cc:1261] T 6ef7ad9ef0e64dc6a802a510327d5142 P ce8da99cd776432a9dd696240ec33e3a [term 1 FOLLOWER]: Refusing update from remote peer 92f8e0876fed4e35a81e6aa5720a8e0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:07:10.265656 3317 consensus_queue.cc:1035] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975d450c614610be98f278819f7ee7" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 37421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:07:10.265856 3314 consensus_queue.cc:1035] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce8da99cd776432a9dd696240ec33e3a" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:07:10.273597 3324 mvcc.cc:205] Tried to move back new op lower bound from 7317520507960193024 to 7317520507881836544. Current Snapshot: MvccSnapshot[applied={T|T < 7317520507960193024}]
I20260812 03:07:10.280707 2555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:07:10.280858 2555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:07:10.283079 2555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e42d60a07d486a80bfab4de27df4da: Sending DeleteTablet for 3 replicas of tablet 6ef7ad9ef0e64dc6a802a510327d5142
I20260812 03:07:10.283775 2754 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ef7ad9ef0e64dc6a802a510327d5142 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:07:10 UTC) from {username='slave'} at 127.0.0.1:60202
I20260812 03:07:10.283775 2975 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ef7ad9ef0e64dc6a802a510327d5142 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:07:10 UTC) from {username='slave'} at 127.0.0.1:59874
I20260812 03:07:10.284237 3340 tablet_replica.cc:330] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b: stopping tablet replica
I20260812 03:07:10.284304 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 2645
I20260812 03:07:10.284350 3171 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ef7ad9ef0e64dc6a802a510327d5142 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:07:10 UTC) from {username='slave'} at 127.0.0.1:47256
I20260812 03:07:10.291777 3340 raft_consensus.cc:2229] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:07:10.291982 3341 tablet_replica.cc:330] T 6ef7ad9ef0e64dc6a802a510327d5142 P ce8da99cd776432a9dd696240ec33e3a: stopping tablet replica
I20260812 03:07:10.292250 3340 raft_consensus.cc:2258] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:07:10.292263 3341 raft_consensus.cc:2229] T 6ef7ad9ef0e64dc6a802a510327d5142 P ce8da99cd776432a9dd696240ec33e3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:07:10.292536 3341 raft_consensus.cc:2258] T 6ef7ad9ef0e64dc6a802a510327d5142 P ce8da99cd776432a9dd696240ec33e3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:07:10.295543 3340 ts_tablet_manager.cc:1905] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:07:10.295581 3341 ts_tablet_manager.cc:1905] T 6ef7ad9ef0e64dc6a802a510327d5142 P ce8da99cd776432a9dd696240ec33e3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:07:10.298133 3341 ts_tablet_manager.cc:1918] T 6ef7ad9ef0e64dc6a802a510327d5142 P ce8da99cd776432a9dd696240ec33e3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:07:10.298229 3341 log.cc:1196] T 6ef7ad9ef0e64dc6a802a510327d5142 P ce8da99cd776432a9dd696240ec33e3a: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-2/wal/wals/6ef7ad9ef0e64dc6a802a510327d5142
I20260812 03:07:10.298256 3340 ts_tablet_manager.cc:1918] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:07:10.298463 3340 log.cc:1196] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786503790637001-5506-0/minicluster-data/ts-1/wal/wals/6ef7ad9ef0e64dc6a802a510327d5142
I20260812 03:07:10.298537 3341 ts_tablet_manager.cc:1939] T 6ef7ad9ef0e64dc6a802a510327d5142 P ce8da99cd776432a9dd696240ec33e3a: Deleting consensus metadata
I20260812 03:07:10.298830 3340 ts_tablet_manager.cc:1939] T 6ef7ad9ef0e64dc6a802a510327d5142 P 92f8e0876fed4e35a81e6aa5720a8e0b: Deleting consensus metadata
I20260812 03:07:10.299731 2534 catalog_manager.cc:4841] TS ce8da99cd776432a9dd696240ec33e3a (127.5.96.131:44749): tablet 6ef7ad9ef0e64dc6a802a510327d5142 (table test-table [id=1e6aa1aed22e439095d4fb0d7d103501]) successfully deleted
I20260812 03:07:10.300041 2530 catalog_manager.cc:4841] TS 92f8e0876fed4e35a81e6aa5720a8e0b (127.5.96.130:37357): tablet 6ef7ad9ef0e64dc6a802a510327d5142 (table test-table [id=1e6aa1aed22e439095d4fb0d7d103501]) successfully deleted
W20260812 03:07:10.301976 2533 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.96.129:37421 (error 108)
I20260812 03:07:10.302788 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 2854
I20260812 03:07:10.302973 3292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:10.302682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:37421 (local address 127.0.0.1:60206)
0812 03:07:10.302818 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:10.302892 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.129:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 03:07:10.309379 2533 catalog_manager.cc:4584] TS 39975d450c614610be98f278819f7ee7 (127.5.96.129:37421): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6ef7ad9ef0e64dc6a802a510327d5142: Network error: Client connection negotiation failed: client connection to 127.5.96.129:37421: connect: Connection refused (error 111)
I20260812 03:07:10.315569 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 3079
I20260812 03:07:10.324818 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 2504
2026-08-12T03:07:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1007 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:07:10 dist-test-slave-6bbx krb5kdc[3347](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:07:10 dist-test-slave-6bbx krb5kdc[3347](info): set up 2 sockets
Aug 12 03:07:10 dist-test-slave-6bbx krb5kdc[3347](info): commencing operation
krb5kdc: starting...
W20260812 03:07:11.468616 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:07:11 dist-test-slave-6bbx krb5kdc[3347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:07:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:07:11.578264 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:38545
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:33203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:38545
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:11.684957 3378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:11.685190 3378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:11.685245 3378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:11.689141 3378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:11.689190 3378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:11.689214 3378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:11.689247 3378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:11.689275 3378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:11.694231 3378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:33203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:38545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:38545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:11.695111 3378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:11.696082 3378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:11.703938 3392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:11.703909 3395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:11.703986 3378 server_base.cc:1034] running on GCE node
W20260812 03:07:11.703920 3390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:11.704651 3378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:11.705629 3378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:11.706785 3378 hybrid_clock.cc:648] HybridClock initialized: now 1786504031706765 us; error 33 us; skew 500 ppm
Aug 12 03:07:11 dist-test-slave-6bbx krb5kdc[3347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504031, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:11.709323 3378 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:07:11.710117 3378 webserver.cc:460] Webserver started at http://127.5.96.190:42829/ using document root <none> and password file <none>
I20260812 03:07:11.710579 3378 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:11.710645 3378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:11.710805 3378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:11.712670 3378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "c1aee46aafb7487a9d488636e13a394b"
format_stamp: "Formatted at 2026-08-12 03:07:11 on dist-test-slave-6bbx"
server_key: "e94132f2107dae1c2eb690ce3b6e6c2b"
server_key_iv: "62fffe63c02f40e91810a44a3aac71cf"
server_key_version: "encryptionkey@0"
I20260812 03:07:11.713146 3378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "c1aee46aafb7487a9d488636e13a394b"
format_stamp: "Formatted at 2026-08-12 03:07:11 on dist-test-slave-6bbx"
server_key: "e94132f2107dae1c2eb690ce3b6e6c2b"
server_key_iv: "62fffe63c02f40e91810a44a3aac71cf"
server_key_version: "encryptionkey@0"
I20260812 03:07:11.716575 3378 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:07:11.718699 3410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:11.719823 3378 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:07:11.719928 3378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "c1aee46aafb7487a9d488636e13a394b"
format_stamp: "Formatted at 2026-08-12 03:07:11 on dist-test-slave-6bbx"
server_key: "e94132f2107dae1c2eb690ce3b6e6c2b"
server_key_iv: "62fffe63c02f40e91810a44a3aac71cf"
server_key_version: "encryptionkey@0"
I20260812 03:07:11.720021 3378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:11.741407 3378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:11.744009 3378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:11.744179 3378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:11.750759 3378 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:38545
I20260812 03:07:11.750773 3490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:38545 every 8 connection(s)
I20260812 03:07:11.751629 3378 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:11.753027 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 3378
I20260812 03:07:11.753167 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:07:11.753468 5506 external_mini_cluster.cc:1428] Setting key c36b18d83a578436049cbae411444601
I20260812 03:07:11.754613 3491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:11.759951 3491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1aee46aafb7487a9d488636e13a394b: Bootstrap starting.
Aug 12 03:07:11 dist-test-slave-6bbx krb5kdc[3347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:07:11.762375 3491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1aee46aafb7487a9d488636e13a394b: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:11.763175 3491 log.cc:824] T 00000000000000000000000000000000 P c1aee46aafb7487a9d488636e13a394b: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:11.765280 3491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1aee46aafb7487a9d488636e13a394b: No bootstrap required, opened a new log
I20260812 03:07:11.766631 3497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:11.754809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43016 (local address 127.5.96.190:38545)
0812 03:07:11.755251 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:11.755266 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:11.755300 (+ 34us) server_negotiation.cc:407] Connection header received
0812 03:07:11.755956 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:11.755974 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:11.756289 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:11.756571 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:11.757349 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:11.758308 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:11.759165 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:11.759620 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:11.759830 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:11.762325 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:11.762330 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:11.762341 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:11.762367 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:11.764079 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:11.764520 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:11.764524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:11.764528 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:11.764629 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:11.764917 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:11.764919 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:11.764920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:11.765276 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:11.765445 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:11.765701 (+ 256us) server_negotiation.cc:299] Negotiation successful
0812 03:07:11.765945 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":155,"threads_started":1}
I20260812 03:07:11.768726 3491 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1aee46aafb7487a9d488636e13a394b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1aee46aafb7487a9d488636e13a394b" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 38545 } }
I20260812 03:07:11.768920 3491 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1aee46aafb7487a9d488636e13a394b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:11.768970 3491 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1aee46aafb7487a9d488636e13a394b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1aee46aafb7487a9d488636e13a394b, State: Initialized, Role: FOLLOWER
I20260812 03:07:11.769344 3491 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1aee46aafb7487a9d488636e13a394b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1aee46aafb7487a9d488636e13a394b" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 38545 } }
I20260812 03:07:11.769464 3491 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1aee46aafb7487a9d488636e13a394b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:07:11.769526 3491 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1aee46aafb7487a9d488636e13a394b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:07:11.769618 3491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1aee46aafb7487a9d488636e13a394b [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:11.770401 3491 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1aee46aafb7487a9d488636e13a394b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1aee46aafb7487a9d488636e13a394b" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 38545 } }
I20260812 03:07:11.770699 3491 leader_election.cc:302] T 00000000000000000000000000000000 P c1aee46aafb7487a9d488636e13a394b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1aee46aafb7487a9d488636e13a394b; no voters:
I20260812 03:07:11.770967 3491 leader_election.cc:288] T 00000000000000000000000000000000 P c1aee46aafb7487a9d488636e13a394b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:07:11.771046 3500 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1aee46aafb7487a9d488636e13a394b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:11.771255 3500 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1aee46aafb7487a9d488636e13a394b [term 1 LEADER]: Becoming Leader. State: Replica: c1aee46aafb7487a9d488636e13a394b, State: Running, Role: LEADER
I20260812 03:07:11.771589 3500 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1aee46aafb7487a9d488636e13a394b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1aee46aafb7487a9d488636e13a394b" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 38545 } }
I20260812 03:07:11.772053 3491 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1aee46aafb7487a9d488636e13a394b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:11.773022 3501 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1aee46aafb7487a9d488636e13a394b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1aee46aafb7487a9d488636e13a394b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1aee46aafb7487a9d488636e13a394b" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 38545 } } }
I20260812 03:07:11.773147 3501 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1aee46aafb7487a9d488636e13a394b [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:11.773476 3502 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1aee46aafb7487a9d488636e13a394b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1aee46aafb7487a9d488636e13a394b. Latest consensus state: current_term: 1 leader_uuid: "c1aee46aafb7487a9d488636e13a394b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1aee46aafb7487a9d488636e13a394b" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 38545 } } }
I20260812 03:07:11.773553 3512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:11.773628 3502 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1aee46aafb7487a9d488636e13a394b [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:11.776503 3512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:11.781608 3512 catalog_manager.cc:1347] Generated new cluster ID: 96082266b9614dafaa924a2656ba3bae
I20260812 03:07:11.781658 3512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:11.802457 3512 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:11.803418 3512 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:11.812615 3512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1aee46aafb7487a9d488636e13a394b: Generated new TSK 0
I20260812 03:07:11.813418 3512 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.96.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
I20260812 03:07:11.848006 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:0
--local_ip_for_outbound_sockets=127.5.96.129
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:38545
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:33203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:11.955914 3533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:11.956133 3533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:11.956177 3533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:11.960050 3533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:11.960096 3533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:11.960166 3533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:07:11.965075 3533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:33203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:38545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:11.965942 3533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:11.966850 3533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:11.975729 3542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:11.975762 3543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:11.975745 3533 server_base.cc:1034] running on GCE node
W20260812 03:07:11.975914 3545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:11.976300 3533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:11.976847 3533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:11.977998 3533 hybrid_clock.cc:648] HybridClock initialized: now 1786504031977976 us; error 33 us; skew 500 ppm
Aug 12 03:07:11 dist-test-slave-6bbx krb5kdc[3347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504031, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:11.980542 3533 init.cc:377] Logged in from keytab as kudu/127.5.96.129@KRBTEST.COM (short username kudu)
I20260812 03:07:11.981308 3533 webserver.cc:460] Webserver started at http://127.5.96.129:43707/ using document root <none> and password file <none>
I20260812 03:07:11.981797 3533 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:11.981861 3533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:11.982035 3533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:11.983986 3533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-0/data/instance:
uuid: "de2acb5a5b04422bbed27f3ec91ec7f0"
format_stamp: "Formatted at 2026-08-12 03:07:11 on dist-test-slave-6bbx"
server_key: "d19b747a58f6742ea252048d10b70ccc"
server_key_iv: "996bbf4839ab0500354a737397e749f7"
server_key_version: "encryptionkey@0"
I20260812 03:07:11.984504 3533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance:
uuid: "de2acb5a5b04422bbed27f3ec91ec7f0"
format_stamp: "Formatted at 2026-08-12 03:07:11 on dist-test-slave-6bbx"
server_key: "d19b747a58f6742ea252048d10b70ccc"
server_key_iv: "996bbf4839ab0500354a737397e749f7"
server_key_version: "encryptionkey@0"
I20260812 03:07:11.987936 3533 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:07:11.990155 3559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:11.991254 3533 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:07:11.991358 3533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "de2acb5a5b04422bbed27f3ec91ec7f0"
format_stamp: "Formatted at 2026-08-12 03:07:11 on dist-test-slave-6bbx"
server_key: "d19b747a58f6742ea252048d10b70ccc"
server_key_iv: "996bbf4839ab0500354a737397e749f7"
server_key_version: "encryptionkey@0"
I20260812 03:07:11.991443 3533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:12.007463 3533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:12.010030 3533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:12.010195 3533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:12.010777 3533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:07:12.011509 3533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:07:12.011555 3533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:12.011590 3533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:07:12.011610 3533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:12.019552 3533 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:38449
I20260812 03:07:12.019582 3725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:38449 every 8 connection(s)
I20260812 03:07:12.020363 3533 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:07:12.023602 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 3533
I20260812 03:07:12.023744 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance
I20260812 03:07:12.024047 5506 external_mini_cluster.cc:1428] Setting key fbb15e5072dc5e0488782ea73a9d26e6
Aug 12 03:07:12 dist-test-slave-6bbx krb5kdc[3347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504031, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:07:12.033023 3497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:12.021949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:37839 (local address 127.5.96.190:38545)
0812 03:07:12.022061 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:12.022064 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:12.022812 (+ 748us) server_negotiation.cc:407] Connection header received
0812 03:07:12.024202 (+ 1390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:12.024206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:12.024267 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:12.024311 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:12.025614 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.026147 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.027216 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.027456 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.027544 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:12.029767 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:12.029772 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:12.029774 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:12.029797 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:12.031124 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:12.031588 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:12.031591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:12.031592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:12.031699 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:12.032017 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:12.032021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:12.032023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:12.032195 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:12.032284 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:12.032735 (+ 451us) server_negotiation.cc:299] Negotiation successful
0812 03:07:12.032861 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
WARNING: no policy specified for kudu/127.5.96.130@KRBTEST.COM; defaulting to no policy
I20260812 03:07:12.033749 3728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:12.022266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:38545 (local address 127.5.96.129:37839)
0812 03:07:12.022669 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:12.022721 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:12.023979 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:12.024639 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:12.024648 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:12.024985 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:12.025480 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:12.025494 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.026269 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.026272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:12.027100 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:12.027110 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.027591 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.027595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:12.027778 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:12.028270 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:12.028285 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:12.029606 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:12.031239 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:12.031245 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:12.031258 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:12.031494 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:12.031796 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:12.031799 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:12.031801 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:12.031931 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:12.032295 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:12.032301 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:12.032511 (+ 210us) client_negotiation.cc:769] Sending connection context
0812 03:07:12.032665 (+ 154us) client_negotiation.cc:240] Negotiation successful
0812 03:07:12.032949 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":101,"threads_started":1}
I20260812 03:07:12.034806 3726 heartbeater.cc:347] Connected to a master server at 127.5.96.190:38545
I20260812 03:07:12.035015 3726 heartbeater.cc:464] Registering TS with master...
I20260812 03:07:12.035550 3726 heartbeater.cc:511] Master 127.5.96.190:38545 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.130@KRBTEST.COM" created.
I20260812 03:07:12.037010 3436 ts_manager.cc:194] Registered new tserver with Master: de2acb5a5b04422bbed27f3ec91ec7f0 (127.5.96.129:38449)
I20260812 03:07:12.038105 3436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.129@KRBTEST.COM'} at 127.5.96.129:37839
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
I20260812 03:07:12.048156 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:0
--local_ip_for_outbound_sockets=127.5.96.130
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:38545
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:33203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:12.153340 3737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:12.153563 3737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:12.153620 3737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:12.157511 3737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:12.157559 3737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:12.157634 3737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:07:12.162567 3737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:33203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:38545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:12.163455 3737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:12.164405 3737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:12.173244 3748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:12.173267 3749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:12.173314 3737 server_base.cc:1034] running on GCE node
W20260812 03:07:12.173267 3753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:12.173831 3737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:12.174435 3737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:12.175597 3737 hybrid_clock.cc:648] HybridClock initialized: now 1786504032175573 us; error 38 us; skew 500 ppm
Aug 12 03:07:12 dist-test-slave-6bbx krb5kdc[3347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504032, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:12.178054 3737 init.cc:377] Logged in from keytab as kudu/127.5.96.130@KRBTEST.COM (short username kudu)
I20260812 03:07:12.178848 3737 webserver.cc:460] Webserver started at http://127.5.96.130:40729/ using document root <none> and password file <none>
I20260812 03:07:12.179342 3737 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:12.179405 3737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:12.179594 3737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:12.181452 3737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-1/data/instance:
uuid: "b2fd326224cc41809d3217d37294093f"
format_stamp: "Formatted at 2026-08-12 03:07:12 on dist-test-slave-6bbx"
server_key: "d822f8d37890231cb3ffbf07cf4df68b"
server_key_iv: "13e860f1f579d48e8814a085013e0b06"
server_key_version: "encryptionkey@0"
I20260812 03:07:12.182013 3737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance:
uuid: "b2fd326224cc41809d3217d37294093f"
format_stamp: "Formatted at 2026-08-12 03:07:12 on dist-test-slave-6bbx"
server_key: "d822f8d37890231cb3ffbf07cf4df68b"
server_key_iv: "13e860f1f579d48e8814a085013e0b06"
server_key_version: "encryptionkey@0"
I20260812 03:07:12.185417 3737 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 03:07:12.187572 3766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:12.188670 3737 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:07:12.188771 3737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "b2fd326224cc41809d3217d37294093f"
format_stamp: "Formatted at 2026-08-12 03:07:12 on dist-test-slave-6bbx"
server_key: "d822f8d37890231cb3ffbf07cf4df68b"
server_key_iv: "13e860f1f579d48e8814a085013e0b06"
server_key_version: "encryptionkey@0"
I20260812 03:07:12.188856 3737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:12.203387 3737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:12.205886 3737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:12.206044 3737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:12.206565 3737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:07:12.207310 3737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:07:12.207358 3737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:12.207407 3737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:07:12.207438 3737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:12.215557 3737 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:46607
I20260812 03:07:12.215585 3943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:46607 every 8 connection(s)
I20260812 03:07:12.216351 3737 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:07:12.223296 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 3737
I20260812 03:07:12.223428 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance
I20260812 03:07:12.223788 5506 external_mini_cluster.cc:1428] Setting key f208d2f952ba093699d5952de567dca1
Aug 12 03:07:12 dist-test-slave-6bbx krb5kdc[3347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504032, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:07:12.228618 3497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:12.217917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:51775 (local address 127.5.96.190:38545)
0812 03:07:12.218047 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:12.218050 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:12.218814 (+ 764us) server_negotiation.cc:407] Connection header received
0812 03:07:12.220056 (+ 1242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:12.220058 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:12.220104 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:12.220172 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:12.221175 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.221679 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.222676 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.222891 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.222958 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:12.225361 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:12.225364 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:12.225366 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:12.225388 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:12.226645 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:12.227201 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:12.227205 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:12.227207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:12.227310 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:12.227592 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:12.227595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:12.227596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:12.227826 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:12.227918 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:12.228333 (+ 415us) server_negotiation.cc:299] Negotiation successful
0812 03:07:12.228460 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 03:07:12.229240 3947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:12.218246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:38545 (local address 127.5.96.130:51775)
0812 03:07:12.218656 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:12.218712 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:12.219878 (+ 1166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:12.220305 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:12.220312 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:12.220622 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:12.221067 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:12.221079 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.221766 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.221768 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:12.222592 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:12.222601 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.222989 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.222991 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:12.223150 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:12.223744 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:12.223763 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:12.225205 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:12.226755 (+ 1550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:12.226761 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:12.226774 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:12.227088 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:12.227393 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:12.227396 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:12.227397 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:12.227517 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:12.227911 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:12.227916 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:12.228116 (+ 200us) client_negotiation.cc:769] Sending connection context
0812 03:07:12.228263 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 03:07:12.228481 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":110,"threads_started":1}
I20260812 03:07:12.230299 3945 heartbeater.cc:347] Connected to a master server at 127.5.96.190:38545
I20260812 03:07:12.230535 3945 heartbeater.cc:464] Registering TS with master...
I20260812 03:07:12.231051 3945 heartbeater.cc:511] Master 127.5.96.190:38545 requested a full tablet report, sending...
I20260812 03:07:12.232102 3436 ts_manager.cc:194] Registered new tserver with Master: b2fd326224cc41809d3217d37294093f (127.5.96.130:46607)
I20260812 03:07:12.232653 3436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.130@KRBTEST.COM'} at 127.5.96.130:51775
WARNING: no policy specified for kudu/127.5.96.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
I20260812 03:07:12.248600 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:0
--local_ip_for_outbound_sockets=127.5.96.131
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:38545
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:33203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:12.355436 3952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:12.355651 3952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:12.355721 3952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:12.359570 3952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:12.359622 3952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:12.359712 3952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:07:12.364648 3952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:33203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:38545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:12.365535 3952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:12.366433 3952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:12.375262 3966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:12.375278 3960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:12.375321 3964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:12.375540 3952 server_base.cc:1034] running on GCE node
I20260812 03:07:12.375918 3952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:12.376469 3952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:12.377609 3952 hybrid_clock.cc:648] HybridClock initialized: now 1786504032377583 us; error 40 us; skew 500 ppm
Aug 12 03:07:12 dist-test-slave-6bbx krb5kdc[3347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504032, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:12.380057 3952 init.cc:377] Logged in from keytab as kudu/127.5.96.131@KRBTEST.COM (short username kudu)
I20260812 03:07:12.380797 3952 webserver.cc:460] Webserver started at http://127.5.96.131:35283/ using document root <none> and password file <none>
I20260812 03:07:12.381285 3952 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:12.381343 3952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:12.381525 3952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:12.383685 3952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-2/data/instance:
uuid: "e55a39e59ad948b7aa1354ee5af0f1c3"
format_stamp: "Formatted at 2026-08-12 03:07:12 on dist-test-slave-6bbx"
server_key: "8a70e9d1f5c10bc271768055c837c2a6"
server_key_iv: "3a0d18f1aeb878ca13211611adf19c7e"
server_key_version: "encryptionkey@0"
I20260812 03:07:12.384230 3952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance:
uuid: "e55a39e59ad948b7aa1354ee5af0f1c3"
format_stamp: "Formatted at 2026-08-12 03:07:12 on dist-test-slave-6bbx"
server_key: "8a70e9d1f5c10bc271768055c837c2a6"
server_key_iv: "3a0d18f1aeb878ca13211611adf19c7e"
server_key_version: "encryptionkey@0"
I20260812 03:07:12.387595 3952 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:07:12.389812 3977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:12.390918 3952 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:07:12.391031 3952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "e55a39e59ad948b7aa1354ee5af0f1c3"
format_stamp: "Formatted at 2026-08-12 03:07:12 on dist-test-slave-6bbx"
server_key: "8a70e9d1f5c10bc271768055c837c2a6"
server_key_iv: "3a0d18f1aeb878ca13211611adf19c7e"
server_key_version: "encryptionkey@0"
I20260812 03:07:12.391141 3952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:12.410177 3952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:12.412633 3952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:12.412796 3952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:12.413311 3952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:07:12.414076 3952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:07:12.414124 3952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:12.414172 3952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:07:12.414198 3952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:12.422034 3952 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:41225
I20260812 03:07:12.422173 4153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:41225 every 8 connection(s)
I20260812 03:07:12.422878 3952 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:07:12.424026 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 3952
I20260812 03:07:12.424118 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance
I20260812 03:07:12.424422 5506 external_mini_cluster.cc:1428] Setting key a05ac3fbdfeb21e85b5caa7fe21de88c
Aug 12 03:07:12 dist-test-slave-6bbx krb5kdc[3347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504032, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:07:12.435174 3497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:12.424612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:52113 (local address 127.5.96.190:38545)
0812 03:07:12.424774 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:12.424778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:12.425812 (+ 1034us) server_negotiation.cc:407] Connection header received
0812 03:07:12.427053 (+ 1241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:12.427056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:12.427111 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:12.427184 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:12.428293 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.428827 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.429921 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.430151 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.430236 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:12.432475 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:12.432478 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:12.432480 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:12.432505 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:12.433598 (+ 1093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:12.434029 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:12.434031 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:12.434032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:12.434097 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:12.434347 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:12.434348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:12.434349 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:12.434502 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:12.434571 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:12.434965 (+ 394us) server_negotiation.cc:299] Negotiation successful
0812 03:07:12.435055 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 03:07:12.435937 4157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:12.425054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:38545 (local address 127.5.96.131:52113)
0812 03:07:12.425514 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:12.425564 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:12.426868 (+ 1304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:12.427350 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:12.427356 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:12.427700 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:12.428155 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:12.428167 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.428942 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.428945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:12.429805 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:12.429815 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.430282 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.430286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:12.430455 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:12.430920 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:12.430934 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:12.432334 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:12.433693 (+ 1359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:12.433697 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:12.433707 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:12.433955 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:12.434156 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:12.434158 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:12.434160 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:12.434282 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:12.434569 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:12.434574 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:12.434788 (+ 214us) client_negotiation.cc:769] Sending connection context
0812 03:07:12.434931 (+ 143us) client_negotiation.cc:240] Negotiation successful
0812 03:07:12.435155 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":122,"threads_started":1}
I20260812 03:07:12.436918 4154 heartbeater.cc:347] Connected to a master server at 127.5.96.190:38545
I20260812 03:07:12.437131 4154 heartbeater.cc:464] Registering TS with master...
I20260812 03:07:12.437633 4154 heartbeater.cc:511] Master 127.5.96.190:38545 requested a full tablet report, sending...
I20260812 03:07:12.438561 3436 ts_manager.cc:194] Registered new tserver with Master: e55a39e59ad948b7aa1354ee5af0f1c3 (127.5.96.131:41225)
I20260812 03:07:12.439064 3436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.131@KRBTEST.COM'} at 127.5.96.131:52113
I20260812 03:07:12.449391 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:07:12.457432 3497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:12.451165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43020 (local address 127.5.96.190:38545)
0812 03:07:12.451282 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:12.451286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:12.451463 (+ 177us) server_negotiation.cc:407] Connection header received
0812 03:07:12.451547 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:12.451549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:12.451585 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:12.451643 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:12.452247 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.452718 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.453438 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.453627 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.453689 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:12.454927 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:12.454931 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:12.454932 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:12.454953 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:12.456052 (+ 1099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:12.456476 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:12.456478 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:12.456479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:12.456544 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:12.456765 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:12.456767 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:12.456768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:12.456937 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:12.457040 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:12.457217 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 03:07:12.457287 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 03:07:12.459966 3436 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:07:12.462824 3436 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:07:12.476475 4176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:12.470585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42648 (local address 127.5.96.129:38449)
0812 03:07:12.470985 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:12.470996 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:12.471021 (+ 25us) server_negotiation.cc:407] Connection header received
0812 03:07:12.471748 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:12.471754 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:12.471967 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:12.472125 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:12.472453 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.473686 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.475238 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.475835 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.475932 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:12.476077 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:12.476228 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 03:07:12.476323 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":104,"threads_started":1}
I20260812 03:07:12.476481 4172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:12.470563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:46607 (local address 127.0.0.1:52500)
0812 03:07:12.471061 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:12.471073 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:12.471155 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:12.472317 (+ 1162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:12.472320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:12.472337 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:12.472446 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:12.472453 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.473863 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.473867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:12.475029 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:12.475039 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.476093 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.476097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:12.476194 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:12.476218 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:07:12.476265 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 03:07:12.476333 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":53,"thread_start_us":82,"threads_started":1}
I20260812 03:07:12.476470 4174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:12.470442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:38449 (local address 127.0.0.1:42648)
0812 03:07:12.470878 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:12.470920 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:12.471038 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:12.472128 (+ 1090us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:12.472133 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:12.472187 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:12.472322 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:12.472330 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.473814 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.473817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:12.475114 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:12.475125 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.475944 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.475948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:12.476054 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:12.476090 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 03:07:12.476195 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 03:07:12.476324 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":329,"thread_start_us":77,"threads_started":1}
I20260812 03:07:12.477303 4177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:12.471104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:41225 (local address 127.0.0.1:33368)
0812 03:07:12.471519 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:12.471530 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:12.471611 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:12.472442 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:12.472445 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:12.472460 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:12.472547 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:12.472553 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.473665 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.473668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:12.475919 (+ 2251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:12.475931 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.476978 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.476982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:12.477081 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:12.477103 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:07:12.477158 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 03:07:12.477205 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":863,"thread_start_us":90,"threads_started":1}
I20260812 03:07:12.477421 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:12.471258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33368 (local address 127.5.96.131:41225)
0812 03:07:12.472035 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:12.472045 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:12.472066 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:07:12.472131 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:12.472135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:12.472301 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:12.472678 (+ 377us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:12.472821 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.473547 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.476063 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.476836 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.476975 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:12.477107 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:12.477188 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 03:07:12.477278 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":155,"threads_started":1}
I20260812 03:07:12.479784 4180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:12.471523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52500 (local address 127.5.96.130:46607)
0812 03:07:12.471850 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:12.471860 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:12.471883 (+ 23us) server_negotiation.cc:407] Connection header received
0812 03:07:12.471947 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:12.471951 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:12.472089 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:12.472336 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:12.472550 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.473720 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.475140 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.475975 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.476552 (+ 577us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:12.476690 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:12.476739 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 03:07:12.476834 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":138,"threads_started":1}
I20260812 03:07:12.479557 4050 tablet_service.cc:1467] Processing CreateTablet for tablet 0e24e0c9f01748958395468a860558dd (DEFAULT_TABLE table=test-table [id=d0f5588d625941d58e3c34ddb8440b46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:07:12.479504 3639 tablet_service.cc:1467] Processing CreateTablet for tablet 0e24e0c9f01748958395468a860558dd (DEFAULT_TABLE table=test-table [id=d0f5588d625941d58e3c34ddb8440b46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:07:12.480552 3639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e24e0c9f01748958395468a860558dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:12.480558 4050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e24e0c9f01748958395468a860558dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:12.482321 3847 tablet_service.cc:1467] Processing CreateTablet for tablet 0e24e0c9f01748958395468a860558dd (DEFAULT_TABLE table=test-table [id=d0f5588d625941d58e3c34ddb8440b46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:07:12.483270 3847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e24e0c9f01748958395468a860558dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:12.486927 4182 tablet_bootstrap.cc:492] T 0e24e0c9f01748958395468a860558dd P de2acb5a5b04422bbed27f3ec91ec7f0: Bootstrap starting.
I20260812 03:07:12.488200 4183 tablet_bootstrap.cc:492] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3: Bootstrap starting.
I20260812 03:07:12.488899 4182 tablet_bootstrap.cc:654] T 0e24e0c9f01748958395468a860558dd P de2acb5a5b04422bbed27f3ec91ec7f0: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:12.489830 4184 tablet_bootstrap.cc:492] T 0e24e0c9f01748958395468a860558dd P b2fd326224cc41809d3217d37294093f: Bootstrap starting.
I20260812 03:07:12.490235 4182 log.cc:824] T 0e24e0c9f01748958395468a860558dd P de2acb5a5b04422bbed27f3ec91ec7f0: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:12.490304 4183 tablet_bootstrap.cc:654] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:12.491394 4183 log.cc:824] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:12.492004 4184 tablet_bootstrap.cc:654] T 0e24e0c9f01748958395468a860558dd P b2fd326224cc41809d3217d37294093f: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:12.492394 4182 tablet_bootstrap.cc:492] T 0e24e0c9f01748958395468a860558dd P de2acb5a5b04422bbed27f3ec91ec7f0: No bootstrap required, opened a new log
I20260812 03:07:12.492587 4182 ts_tablet_manager.cc:1397] T 0e24e0c9f01748958395468a860558dd P de2acb5a5b04422bbed27f3ec91ec7f0: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260812 03:07:12.493211 4184 log.cc:824] T 0e24e0c9f01748958395468a860558dd P b2fd326224cc41809d3217d37294093f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:12.494874 4184 tablet_bootstrap.cc:492] T 0e24e0c9f01748958395468a860558dd P b2fd326224cc41809d3217d37294093f: No bootstrap required, opened a new log
I20260812 03:07:12.495052 4184 ts_tablet_manager.cc:1397] T 0e24e0c9f01748958395468a860558dd P b2fd326224cc41809d3217d37294093f: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260812 03:07:12.495198 4183 tablet_bootstrap.cc:492] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3: No bootstrap required, opened a new log
I20260812 03:07:12.495366 4183 ts_tablet_manager.cc:1397] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 03:07:12.496412 4182 raft_consensus.cc:348] T 0e24e0c9f01748958395468a860558dd P de2acb5a5b04422bbed27f3ec91ec7f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2acb5a5b04422bbed27f3ec91ec7f0" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 38449 } } peers { permanent_uuid: "b2fd326224cc41809d3217d37294093f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 46607 } } peers { permanent_uuid: "e55a39e59ad948b7aa1354ee5af0f1c3" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 41225 } }
I20260812 03:07:12.496654 4182 raft_consensus.cc:374] T 0e24e0c9f01748958395468a860558dd P de2acb5a5b04422bbed27f3ec91ec7f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:12.496696 4182 raft_consensus.cc:729] T 0e24e0c9f01748958395468a860558dd P de2acb5a5b04422bbed27f3ec91ec7f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de2acb5a5b04422bbed27f3ec91ec7f0, State: Initialized, Role: FOLLOWER
I20260812 03:07:12.497125 4182 consensus_queue.cc:260] T 0e24e0c9f01748958395468a860558dd P de2acb5a5b04422bbed27f3ec91ec7f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2acb5a5b04422bbed27f3ec91ec7f0" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 38449 } } peers { permanent_uuid: "b2fd326224cc41809d3217d37294093f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 46607 } } peers { permanent_uuid: "e55a39e59ad948b7aa1354ee5af0f1c3" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 41225 } }
I20260812 03:07:12.497797 3726 heartbeater.cc:503] Master 127.5.96.190:38545 was elected leader, sending a full tablet report...
I20260812 03:07:12.498097 4182 ts_tablet_manager.cc:1428] T 0e24e0c9f01748958395468a860558dd P de2acb5a5b04422bbed27f3ec91ec7f0: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260812 03:07:12.498034 4184 raft_consensus.cc:348] T 0e24e0c9f01748958395468a860558dd P b2fd326224cc41809d3217d37294093f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2acb5a5b04422bbed27f3ec91ec7f0" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 38449 } } peers { permanent_uuid: "b2fd326224cc41809d3217d37294093f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 46607 } } peers { permanent_uuid: "e55a39e59ad948b7aa1354ee5af0f1c3" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 41225 } }
I20260812 03:07:12.498267 4184 raft_consensus.cc:374] T 0e24e0c9f01748958395468a860558dd P b2fd326224cc41809d3217d37294093f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:12.498484 4184 raft_consensus.cc:729] T 0e24e0c9f01748958395468a860558dd P b2fd326224cc41809d3217d37294093f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2fd326224cc41809d3217d37294093f, State: Initialized, Role: FOLLOWER
I20260812 03:07:12.498927 4184 consensus_queue.cc:260] T 0e24e0c9f01748958395468a860558dd P b2fd326224cc41809d3217d37294093f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2acb5a5b04422bbed27f3ec91ec7f0" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 38449 } } peers { permanent_uuid: "b2fd326224cc41809d3217d37294093f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 46607 } } peers { permanent_uuid: "e55a39e59ad948b7aa1354ee5af0f1c3" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 41225 } }
I20260812 03:07:12.499568 3945 heartbeater.cc:503] Master 127.5.96.190:38545 was elected leader, sending a full tablet report...
I20260812 03:07:12.499491 4183 raft_consensus.cc:348] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2acb5a5b04422bbed27f3ec91ec7f0" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 38449 } } peers { permanent_uuid: "b2fd326224cc41809d3217d37294093f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 46607 } } peers { permanent_uuid: "e55a39e59ad948b7aa1354ee5af0f1c3" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 41225 } }
I20260812 03:07:12.499895 4184 ts_tablet_manager.cc:1428] T 0e24e0c9f01748958395468a860558dd P b2fd326224cc41809d3217d37294093f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 03:07:12.499886 4183 raft_consensus.cc:374] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:12.500020 4183 raft_consensus.cc:729] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e55a39e59ad948b7aa1354ee5af0f1c3, State: Initialized, Role: FOLLOWER
I20260812 03:07:12.500478 4183 consensus_queue.cc:260] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2acb5a5b04422bbed27f3ec91ec7f0" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 38449 } } peers { permanent_uuid: "b2fd326224cc41809d3217d37294093f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 46607 } } peers { permanent_uuid: "e55a39e59ad948b7aa1354ee5af0f1c3" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 41225 } }
I20260812 03:07:12.501081 4154 heartbeater.cc:503] Master 127.5.96.190:38545 was elected leader, sending a full tablet report...
I20260812 03:07:12.501614 4183 ts_tablet_manager.cc:1428] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260812 03:07:12.521379 3727 tablet.cc:2369] T 0e24e0c9f01748958395468a860558dd P de2acb5a5b04422bbed27f3ec91ec7f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:07:12.667871 4195 raft_consensus.cc:482] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:07:12.668081 4195 raft_consensus.cc:504] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2acb5a5b04422bbed27f3ec91ec7f0" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 38449 } } peers { permanent_uuid: "b2fd326224cc41809d3217d37294093f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 46607 } } peers { permanent_uuid: "e55a39e59ad948b7aa1354ee5af0f1c3" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 41225 } }
I20260812 03:07:12.669272 4195 leader_election.cc:288] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de2acb5a5b04422bbed27f3ec91ec7f0 (127.5.96.129:38449), b2fd326224cc41809d3217d37294093f (127.5.96.130:46607)
I20260812 03:07:12.672906 4157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:12.669480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:38449 (local address 127.5.96.131:60487)
0812 03:07:12.669791 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:12.669806 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:12.669904 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:12.670130 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:12.670133 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:12.670149 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:12.670266 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:12.670272 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.670971 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.670975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:12.671887 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:12.671897 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.672494 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.672497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:12.672604 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:12.672627 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:07:12.672694 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 03:07:12.672771 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":18}
I20260812 03:07:12.672906 4176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:12.669582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:60487 (local address 127.5.96.129:38449)
0812 03:07:12.669724 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:12.669729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:12.669876 (+ 147us) server_negotiation.cc:407] Connection header received
0812 03:07:12.670007 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:12.670010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:12.670051 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:12.670110 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:12.670374 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.670866 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.671999 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.672390 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.672493 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:12.672550 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:12.672733 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 03:07:12.672803 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 03:07:12.673058 4180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:12.669639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:33319 (local address 127.5.96.130:46607)
0812 03:07:12.669784 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:12.669789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:12.669995 (+ 206us) server_negotiation.cc:407] Connection header received
0812 03:07:12.670117 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:12.670120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:12.670159 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:12.670218 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:12.670473 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.671019 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.672143 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.672543 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.672674 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:12.672724 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:12.672885 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 03:07:12.672947 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:07:12.673166 4198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:12.669527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:46607 (local address 127.5.96.131:33319)
0812 03:07:12.669914 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:12.669925 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:12.670020 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:12.670236 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:12.670238 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:12.670261 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:12.670376 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:12.670381 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.671125 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.671128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:12.672030 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:12.672040 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.672674 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.672678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:12.672776 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:12.672798 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:07:12.672862 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 03:07:12.672924 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":18,"thread_start_us":109,"threads_started":1}
I20260812 03:07:12.673640 3669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e24e0c9f01748958395468a860558dd" candidate_uuid: "e55a39e59ad948b7aa1354ee5af0f1c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de2acb5a5b04422bbed27f3ec91ec7f0" is_pre_election: true
W20260812 03:07:12.673764 4156 tablet.cc:2369] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:07:12.673830 3875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e24e0c9f01748958395468a860558dd" candidate_uuid: "e55a39e59ad948b7aa1354ee5af0f1c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2fd326224cc41809d3217d37294093f" is_pre_election: true
I20260812 03:07:12.673990 3669 raft_consensus.cc:2454] T 0e24e0c9f01748958395468a860558dd P de2acb5a5b04422bbed27f3ec91ec7f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e55a39e59ad948b7aa1354ee5af0f1c3 in term 0.
I20260812 03:07:12.674103 3875 raft_consensus.cc:2454] T 0e24e0c9f01748958395468a860558dd P b2fd326224cc41809d3217d37294093f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e55a39e59ad948b7aa1354ee5af0f1c3 in term 0.
I20260812 03:07:12.674559 3980 leader_election.cc:302] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2fd326224cc41809d3217d37294093f, e55a39e59ad948b7aa1354ee5af0f1c3; no voters:
I20260812 03:07:12.674856 4195 raft_consensus.cc:2781] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:07:12.674934 4195 raft_consensus.cc:482] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:07:12.674981 4195 raft_consensus.cc:3037] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:12.676028 4195 raft_consensus.cc:504] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2acb5a5b04422bbed27f3ec91ec7f0" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 38449 } } peers { permanent_uuid: "b2fd326224cc41809d3217d37294093f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 46607 } } peers { permanent_uuid: "e55a39e59ad948b7aa1354ee5af0f1c3" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 41225 } }
I20260812 03:07:12.676327 4195 leader_election.cc:288] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3 [CANDIDATE]: Term 1 election: Requested vote from peers de2acb5a5b04422bbed27f3ec91ec7f0 (127.5.96.129:38449), b2fd326224cc41809d3217d37294093f (127.5.96.130:46607)
I20260812 03:07:12.676631 3875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e24e0c9f01748958395468a860558dd" candidate_uuid: "e55a39e59ad948b7aa1354ee5af0f1c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2fd326224cc41809d3217d37294093f"
I20260812 03:07:12.676656 3669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e24e0c9f01748958395468a860558dd" candidate_uuid: "e55a39e59ad948b7aa1354ee5af0f1c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de2acb5a5b04422bbed27f3ec91ec7f0"
I20260812 03:07:12.676738 3875 raft_consensus.cc:3037] T 0e24e0c9f01748958395468a860558dd P b2fd326224cc41809d3217d37294093f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:12.676764 3669 raft_consensus.cc:3037] T 0e24e0c9f01748958395468a860558dd P de2acb5a5b04422bbed27f3ec91ec7f0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:12.677685 3875 raft_consensus.cc:2454] T 0e24e0c9f01748958395468a860558dd P b2fd326224cc41809d3217d37294093f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e55a39e59ad948b7aa1354ee5af0f1c3 in term 1.
I20260812 03:07:12.677762 3669 raft_consensus.cc:2454] T 0e24e0c9f01748958395468a860558dd P de2acb5a5b04422bbed27f3ec91ec7f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e55a39e59ad948b7aa1354ee5af0f1c3 in term 1.
I20260812 03:07:12.678058 3980 leader_election.cc:302] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2fd326224cc41809d3217d37294093f, e55a39e59ad948b7aa1354ee5af0f1c3; no voters:
I20260812 03:07:12.678236 4195 raft_consensus.cc:2781] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:12.678435 4195 raft_consensus.cc:686] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3 [term 1 LEADER]: Becoming Leader. State: Replica: e55a39e59ad948b7aa1354ee5af0f1c3, State: Running, Role: LEADER
I20260812 03:07:12.678750 4195 consensus_queue.cc:237] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2acb5a5b04422bbed27f3ec91ec7f0" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 38449 } } peers { permanent_uuid: "b2fd326224cc41809d3217d37294093f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 46607 } } peers { permanent_uuid: "e55a39e59ad948b7aa1354ee5af0f1c3" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 41225 } }
I20260812 03:07:12.682113 3436 catalog_manager.cc:5495] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e55a39e59ad948b7aa1354ee5af0f1c3 (127.5.96.131). New cstate: current_term: 1 leader_uuid: "e55a39e59ad948b7aa1354ee5af0f1c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2acb5a5b04422bbed27f3ec91ec7f0" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 38449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2fd326224cc41809d3217d37294093f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e55a39e59ad948b7aa1354ee5af0f1c3" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 41225 } health_report { overall_health: HEALTHY } } }
I20260812 03:07:12.706660 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:12.703391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33376 (local address 127.5.96.131:41225)
0812 03:07:12.703512 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:12.703517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:12.703530 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:07:12.703715 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:12.703718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:12.703776 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:12.703860 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:07:12.704091 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.704599 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.705456 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.705659 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.705743 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:12.705957 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:07:12.706313 (+ 356us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:07:12.706413 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:12.706474 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 03:07:12.706505 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 03:07:12.711936 3875 raft_consensus.cc:1261] T 0e24e0c9f01748958395468a860558dd P b2fd326224cc41809d3217d37294093f [term 1 FOLLOWER]: Refusing update from remote peer e55a39e59ad948b7aa1354ee5af0f1c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:07:12.711936 3669 raft_consensus.cc:1261] T 0e24e0c9f01748958395468a860558dd P de2acb5a5b04422bbed27f3ec91ec7f0 [term 1 FOLLOWER]: Refusing update from remote peer e55a39e59ad948b7aa1354ee5af0f1c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:07:12.712505 4202 consensus_queue.cc:1035] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de2acb5a5b04422bbed27f3ec91ec7f0" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:07:12.712666 4195 consensus_queue.cc:1035] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fd326224cc41809d3217d37294093f" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 03:07:12.717299 3946 tablet.cc:2369] T 0e24e0c9f01748958395468a860558dd P b2fd326224cc41809d3217d37294093f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:07:12.719911 4206 mvcc.cc:205] Tried to move back new op lower bound from 7317520517981442048 to 7317520517854441472. Current Snapshot: MvccSnapshot[applied={T|T < 7317520517981442048}]
I20260812 03:07:12.720069 4205 mvcc.cc:205] Tried to move back new op lower bound from 7317520517981442048 to 7317520517854441472. Current Snapshot: MvccSnapshot[applied={T|T < 7317520517981442048}]
I20260812 03:07:12.729200 3436 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:07:12.729369 3436 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:07:12.731716 3436 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c1aee46aafb7487a9d488636e13a394b: Sending DeleteTablet for 3 replicas of tablet 0e24e0c9f01748958395468a860558dd
I20260812 03:07:12.732411 3639 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e24e0c9f01748958395468a860558dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:07:12 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:42648
I20260812 03:07:12.732463 3847 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e24e0c9f01748958395468a860558dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:07:12 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:52500
I20260812 03:07:12.732725 4050 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e24e0c9f01748958395468a860558dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:07:12 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:33368
I20260812 03:07:12.732962 4221 tablet_replica.cc:330] T 0e24e0c9f01748958395468a860558dd P b2fd326224cc41809d3217d37294093f: stopping tablet replica
I20260812 03:07:12.733243 4221 raft_consensus.cc:2229] T 0e24e0c9f01748958395468a860558dd P b2fd326224cc41809d3217d37294093f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:07:12.733433 4227 tablet_replica.cc:330] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3: stopping tablet replica
I20260812 03:07:12.733531 4221 raft_consensus.cc:2258] T 0e24e0c9f01748958395468a860558dd P b2fd326224cc41809d3217d37294093f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:07:12.733551 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 3533
I20260812 03:07:12.733685 4227 raft_consensus.cc:2229] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:07:12.734032 4227 raft_consensus.cc:2258] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:07:12.735334 4221 ts_tablet_manager.cc:1905] T 0e24e0c9f01748958395468a860558dd P b2fd326224cc41809d3217d37294093f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:07:12.735334 4227 ts_tablet_manager.cc:1905] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:07:12.737916 4221 ts_tablet_manager.cc:1918] T 0e24e0c9f01748958395468a860558dd P b2fd326224cc41809d3217d37294093f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:07:12.737916 4227 ts_tablet_manager.cc:1918] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:07:12.738004 4227 log.cc:1196] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-2/wal/wals/0e24e0c9f01748958395468a860558dd
I20260812 03:07:12.738004 4221 log.cc:1196] T 0e24e0c9f01748958395468a860558dd P b2fd326224cc41809d3217d37294093f: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786503790637001-5506-0/minicluster-data/ts-1/wal/wals/0e24e0c9f01748958395468a860558dd
I20260812 03:07:12.738301 4227 ts_tablet_manager.cc:1939] T 0e24e0c9f01748958395468a860558dd P e55a39e59ad948b7aa1354ee5af0f1c3: Deleting consensus metadata
I20260812 03:07:12.738301 4221 ts_tablet_manager.cc:1939] T 0e24e0c9f01748958395468a860558dd P b2fd326224cc41809d3217d37294093f: Deleting consensus metadata
I20260812 03:07:12.739351 3413 catalog_manager.cc:4841] TS b2fd326224cc41809d3217d37294093f (127.5.96.130:46607): tablet 0e24e0c9f01748958395468a860558dd (table test-table [id=d0f5588d625941d58e3c34ddb8440b46]) successfully deleted
I20260812 03:07:12.739501 3413 catalog_manager.cc:4841] TS e55a39e59ad948b7aa1354ee5af0f1c3 (127.5.96.131:41225): tablet 0e24e0c9f01748958395468a860558dd (table test-table [id=d0f5588d625941d58e3c34ddb8440b46]) successfully deleted
W20260812 03:07:12.750356 3414 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.96.129:38449 (error 108)
I20260812 03:07:12.750633 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 3737
I20260812 03:07:12.755211 4177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:12.752836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:38449 (local address 127.0.0.1:42662)
0812 03:07:12.755065 (+ 2229us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:12.755136 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.129:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2144}
W20260812 03:07:12.755374 3414 catalog_manager.cc:4584] TS de2acb5a5b04422bbed27f3ec91ec7f0 (127.5.96.129:38449): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0e24e0c9f01748958395468a860558dd: Network error: Client connection negotiation failed: client connection to 127.5.96.129:38449: connect: Connection refused (error 111)
I20260812 03:07:12.761214 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 3952
I20260812 03:07:12.774071 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 3378
2026-08-12T03:07:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2448 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T03:07:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:12Z Disabled control of system clock
I20260812 03:07:12.805748 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:37285
--webserver_interface=127.5.96.190
--webserver_port=0
--builtin_ntp_servers=127.5.96.148:39653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:37285
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:07:12.912951 4234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:12.913177 4234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:12.913210 4234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:12.916975 4234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:12.917016 4234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:12.917029 4234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:12.917043 4234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:12.917063 4234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:12.921535 4234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:39653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:37285
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:37285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:12.922394 4234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:12.923573 4234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:12.931380 4239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:12.931344 4241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:12.931366 4245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:12.931689 4234 server_base.cc:1034] running on GCE node
I20260812 03:07:12.932212 4234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:12.933399 4234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:12.934562 4234 hybrid_clock.cc:648] HybridClock initialized: now 1786504032934536 us; error 43 us; skew 500 ppm
I20260812 03:07:12.936530 4234 webserver.cc:460] Webserver started at http://127.5.96.190:41267/ using document root <none> and password file <none>
I20260812 03:07:12.937203 4234 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:12.937284 4234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:12.937515 4234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:12.939827 4234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "7d7b37fab0d4427282f56b4ebff71a26"
format_stamp: "Formatted at 2026-08-12 03:07:12 on dist-test-slave-6bbx"
server_key: "202feffef690ce6ce12516b401bcf057"
server_key_iv: "9dd8b376af8e5cca30768d98cbb36149"
server_key_version: "encryptionkey@0"
I20260812 03:07:12.940357 4234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "7d7b37fab0d4427282f56b4ebff71a26"
format_stamp: "Formatted at 2026-08-12 03:07:12 on dist-test-slave-6bbx"
server_key: "202feffef690ce6ce12516b401bcf057"
server_key_iv: "9dd8b376af8e5cca30768d98cbb36149"
server_key_version: "encryptionkey@0"
I20260812 03:07:12.944092 4234 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:07:12.946408 4255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:12.947517 4234 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:07:12.947628 4234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "7d7b37fab0d4427282f56b4ebff71a26"
format_stamp: "Formatted at 2026-08-12 03:07:12 on dist-test-slave-6bbx"
server_key: "202feffef690ce6ce12516b401bcf057"
server_key_iv: "9dd8b376af8e5cca30768d98cbb36149"
server_key_version: "encryptionkey@0"
I20260812 03:07:12.947767 4234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:12.980311 4234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:12.980919 4234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:12.981076 4234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:12.987586 4234 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:37285
I20260812 03:07:12.987599 4327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:37285 every 8 connection(s)
I20260812 03:07:12.988478 4234 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:12.991417 4329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:12.991684 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 4234
I20260812 03:07:12.991808 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:07:12.992129 5506 external_mini_cluster.cc:1428] Setting key 0a05c5d4dcbae446cb0f3c9e2b96da7d
I20260812 03:07:12.998155 4329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d7b37fab0d4427282f56b4ebff71a26: Bootstrap starting.
I20260812 03:07:13.000583 4329 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d7b37fab0d4427282f56b4ebff71a26: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:13.001108 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:12.993452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44502 (local address 127.5.96.190:37285)
0812 03:07:12.994034 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:12.994050 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:12.994085 (+ 35us) server_negotiation.cc:407] Connection header received
0812 03:07:12.994754 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:12.994772 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:12.995148 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:12.995416 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:12.995781 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.996800 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.997956 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:12.998446 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:12.998559 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:12.999402 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:12.999407 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:12.999419 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:12.999831 (+ 412us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:12.999886 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:12.999906 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:12.999979 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:13.000120 (+ 141us) server_negotiation.cc:299] Negotiation successful
0812 03:07:13.000372 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":121,"threads_started":1}
I20260812 03:07:13.001325 4329 log.cc:824] T 00000000000000000000000000000000 P 7d7b37fab0d4427282f56b4ebff71a26: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:13.003171 4329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d7b37fab0d4427282f56b4ebff71a26: No bootstrap required, opened a new log
I20260812 03:07:13.006569 4329 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d7b37fab0d4427282f56b4ebff71a26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d7b37fab0d4427282f56b4ebff71a26" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37285 } }
I20260812 03:07:13.006781 4329 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d7b37fab0d4427282f56b4ebff71a26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:13.006839 4329 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d7b37fab0d4427282f56b4ebff71a26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d7b37fab0d4427282f56b4ebff71a26, State: Initialized, Role: FOLLOWER
I20260812 03:07:13.007236 4329 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d7b37fab0d4427282f56b4ebff71a26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d7b37fab0d4427282f56b4ebff71a26" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37285 } }
I20260812 03:07:13.007365 4329 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d7b37fab0d4427282f56b4ebff71a26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:07:13.007434 4329 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d7b37fab0d4427282f56b4ebff71a26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:07:13.007508 4329 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d7b37fab0d4427282f56b4ebff71a26 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:13.008314 4329 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d7b37fab0d4427282f56b4ebff71a26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d7b37fab0d4427282f56b4ebff71a26" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37285 } }
I20260812 03:07:13.008632 4329 leader_election.cc:302] T 00000000000000000000000000000000 P 7d7b37fab0d4427282f56b4ebff71a26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d7b37fab0d4427282f56b4ebff71a26; no voters:
I20260812 03:07:13.008950 4329 leader_election.cc:288] T 00000000000000000000000000000000 P 7d7b37fab0d4427282f56b4ebff71a26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:07:13.009038 4337 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d7b37fab0d4427282f56b4ebff71a26 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:13.009197 4337 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d7b37fab0d4427282f56b4ebff71a26 [term 1 LEADER]: Becoming Leader. State: Replica: 7d7b37fab0d4427282f56b4ebff71a26, State: Running, Role: LEADER
I20260812 03:07:13.009485 4337 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d7b37fab0d4427282f56b4ebff71a26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d7b37fab0d4427282f56b4ebff71a26" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37285 } }
I20260812 03:07:13.010143 4329 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d7b37fab0d4427282f56b4ebff71a26 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:13.011109 4340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d7b37fab0d4427282f56b4ebff71a26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d7b37fab0d4427282f56b4ebff71a26. Latest consensus state: current_term: 1 leader_uuid: "7d7b37fab0d4427282f56b4ebff71a26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d7b37fab0d4427282f56b4ebff71a26" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37285 } } }
I20260812 03:07:13.011138 4339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d7b37fab0d4427282f56b4ebff71a26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d7b37fab0d4427282f56b4ebff71a26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d7b37fab0d4427282f56b4ebff71a26" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 37285 } } }
I20260812 03:07:13.011233 4340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d7b37fab0d4427282f56b4ebff71a26 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:13.011245 4339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d7b37fab0d4427282f56b4ebff71a26 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:13.011621 4356 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:13.014499 4356 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:13.019106 4356 catalog_manager.cc:1347] Generated new cluster ID: be5a83bfa950471086815f71eadfd850
I20260812 03:07:13.019167 4356 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:13.028558 4356 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:13.029328 4356 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:13.041075 4356 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d7b37fab0d4427282f56b4ebff71a26: Generated new TSK 0
I20260812 03:07:13.041620 4356 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:07:13.047580 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:0
--local_ip_for_outbound_sockets=127.5.96.129
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:37285
--builtin_ntp_servers=127.5.96.148:39653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:07:13.197355 4372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:13.197583 4372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:13.197623 4372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:13.201510 4372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:13.201556 4372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:13.201618 4372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:07:13.206190 4372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:39653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:37285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:13.207106 4372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:13.208387 4372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:13.217429 4381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:13.217440 4382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:13.217470 4384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:13.217707 4372 server_base.cc:1034] running on GCE node
I20260812 03:07:13.218084 4372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:13.218642 4372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:13.219781 4372 hybrid_clock.cc:648] HybridClock initialized: now 1786504033219766 us; error 25 us; skew 500 ppm
I20260812 03:07:13.221441 4372 webserver.cc:460] Webserver started at http://127.5.96.129:44527/ using document root <none> and password file <none>
I20260812 03:07:13.221956 4372 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:13.222015 4372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:13.222222 4372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:13.224186 4372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-0/data/instance:
uuid: "c4d9df419e794feeae0a47309a028283"
format_stamp: "Formatted at 2026-08-12 03:07:13 on dist-test-slave-6bbx"
server_key: "4a63c3029688f76109a7cc3a5fe12d63"
server_key_iv: "35feb447987510d22435ff3e2f5f9fe9"
server_key_version: "encryptionkey@0"
I20260812 03:07:13.224684 4372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance:
uuid: "c4d9df419e794feeae0a47309a028283"
format_stamp: "Formatted at 2026-08-12 03:07:13 on dist-test-slave-6bbx"
server_key: "4a63c3029688f76109a7cc3a5fe12d63"
server_key_iv: "35feb447987510d22435ff3e2f5f9fe9"
server_key_version: "encryptionkey@0"
I20260812 03:07:13.227980 4372 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:07:13.230120 4392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:13.231189 4372 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:07:13.231297 4372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "c4d9df419e794feeae0a47309a028283"
format_stamp: "Formatted at 2026-08-12 03:07:13 on dist-test-slave-6bbx"
server_key: "4a63c3029688f76109a7cc3a5fe12d63"
server_key_iv: "35feb447987510d22435ff3e2f5f9fe9"
server_key_version: "encryptionkey@0"
I20260812 03:07:13.231385 4372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:13.275822 4372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:13.276472 4372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:13.276639 4372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:13.277204 4372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:07:13.278118 4372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:07:13.278167 4372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:13.278209 4372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:07:13.278239 4372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:13.286304 4372 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:35319
I20260812 03:07:13.286338 4579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:35319 every 8 connection(s)
I20260812 03:07:13.287125 4372 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:07:13.295122 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 4372
I20260812 03:07:13.295254 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance
I20260812 03:07:13.295596 5506 external_mini_cluster.cc:1428] Setting key 6049e928bca2dd4b238de61075cb0749
I20260812 03:07:13.295637 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:13.288812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:54531 (local address 127.5.96.190:37285)
0812 03:07:13.288947 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:13.288951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:13.290028 (+ 1077us) server_negotiation.cc:407] Connection header received
0812 03:07:13.291467 (+ 1439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:13.291469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:13.291514 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:13.291575 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:13.292417 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.292942 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.293971 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.294170 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.294235 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:13.295208 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:13.295212 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:13.295215 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:13.295229 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:13.295251 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:13.295262 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:13.295329 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:13.295474 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 03:07:13.295526 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:07:13.296520 4589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:13.289161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37285 (local address 127.5.96.129:54531)
0812 03:07:13.289874 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:13.289925 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:13.291270 (+ 1345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:13.291758 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:13.291765 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:13.291847 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:13.292305 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:13.292317 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.293032 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.293034 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:13.293886 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:13.293895 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.294272 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.294275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:13.294406 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:13.294931 (+ 525us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:07:13.294933 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:13.294984 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:07:13.295059 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:13.295335 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:13.295345 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:13.295351 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:07:13.295477 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 03:07:13.295786 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":109,"threads_started":1}
I20260812 03:07:13.297585 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:0
--local_ip_for_outbound_sockets=127.5.96.130
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:37285
--builtin_ntp_servers=127.5.96.148:39653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 03:07:13.297832 4587 heartbeater.cc:347] Connected to a master server at 127.5.96.190:37285
I20260812 03:07:13.298029 4587 heartbeater.cc:464] Registering TS with master...
I20260812 03:07:13.298527 4587 heartbeater.cc:511] Master 127.5.96.190:37285 requested a full tablet report, sending...
I20260812 03:07:13.300118 4282 ts_manager.cc:194] Registered new tserver with Master: c4d9df419e794feeae0a47309a028283 (127.5.96.129:35319)
I20260812 03:07:13.301540 4282 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.96.129:54531
W20260812 03:07:13.405841 4591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:13.406077 4591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:13.406112 4591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:13.409943 4591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:13.409986 4591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:13.410034 4591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:07:13.414527 4591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:39653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:37285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:13.415426 4591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:13.416677 4591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:13.425819 4597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:13.425891 4591 server_base.cc:1034] running on GCE node
W20260812 03:07:13.425832 4600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:13.425828 4602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:13.426465 4591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:13.427045 4591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:13.428200 4591 hybrid_clock.cc:648] HybridClock initialized: now 1786504033428178 us; error 32 us; skew 500 ppm
I20260812 03:07:13.429858 4591 webserver.cc:460] Webserver started at http://127.5.96.130:37013/ using document root <none> and password file <none>
I20260812 03:07:13.430384 4591 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:13.430450 4591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:13.430634 4591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:13.432611 4591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-1/data/instance:
uuid: "f0b03a5e593d4ba6a94193297706a46e"
format_stamp: "Formatted at 2026-08-12 03:07:13 on dist-test-slave-6bbx"
server_key: "69c3313be2bd5de0e50a93e4942b33b2"
server_key_iv: "392b66e871c1a7af819fc75a10a2cd26"
server_key_version: "encryptionkey@0"
I20260812 03:07:13.433128 4591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance:
uuid: "f0b03a5e593d4ba6a94193297706a46e"
format_stamp: "Formatted at 2026-08-12 03:07:13 on dist-test-slave-6bbx"
server_key: "69c3313be2bd5de0e50a93e4942b33b2"
server_key_iv: "392b66e871c1a7af819fc75a10a2cd26"
server_key_version: "encryptionkey@0"
I20260812 03:07:13.436657 4591 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:07:13.438920 4620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:13.440083 4591 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:07:13.440187 4591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "f0b03a5e593d4ba6a94193297706a46e"
format_stamp: "Formatted at 2026-08-12 03:07:13 on dist-test-slave-6bbx"
server_key: "69c3313be2bd5de0e50a93e4942b33b2"
server_key_iv: "392b66e871c1a7af819fc75a10a2cd26"
server_key_version: "encryptionkey@0"
I20260812 03:07:13.440285 4591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:13.464694 4591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:13.465288 4591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:13.465447 4591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:13.465969 4591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:07:13.466832 4591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:07:13.466881 4591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:13.466929 4591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:07:13.466958 4591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:13.475076 4591 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:33531
I20260812 03:07:13.475113 4782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:33531 every 8 connection(s)
I20260812 03:07:13.475911 4591 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:07:13.484127 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:13.477583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:50185 (local address 127.5.96.190:37285)
0812 03:07:13.477717 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:13.477731 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:13.478684 (+ 953us) server_negotiation.cc:407] Connection header received
0812 03:07:13.480040 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:13.480043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:13.480088 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:13.480158 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:13.480955 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.481464 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.482489 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.482688 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.482762 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:13.483763 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:13.483767 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:13.483769 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:13.483780 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:13.483798 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:13.483806 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:13.483861 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:13.483970 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 03:07:13.484017 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 03:07:13.484850 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 4591
I20260812 03:07:13.484973 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance
I20260812 03:07:13.485013 4786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:13.477937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37285 (local address 127.5.96.130:50185)
0812 03:07:13.478540 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:13.478578 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:13.479864 (+ 1286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:13.480296 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:13.480303 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:13.480383 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:13.480847 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:13.480860 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.481548 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.481550 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:13.482403 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:13.482412 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.482796 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.482798 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:13.482955 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:13.483492 (+ 537us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:07:13.483494 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:13.483531 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:07:13.483589 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:13.483869 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:13.483874 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:13.483878 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:07:13.483964 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 03:07:13.484162 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":104,"threads_started":1}
I20260812 03:07:13.485314 5506 external_mini_cluster.cc:1428] Setting key 43e91b11c89777cacf20b9cebe011998
I20260812 03:07:13.486137 4784 heartbeater.cc:347] Connected to a master server at 127.5.96.190:37285
I20260812 03:07:13.486339 4784 heartbeater.cc:464] Registering TS with master...
I20260812 03:07:13.486898 4784 heartbeater.cc:511] Master 127.5.96.190:37285 requested a full tablet report, sending...
I20260812 03:07:13.487229 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:0
--local_ip_for_outbound_sockets=127.5.96.131
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:37285
--builtin_ntp_servers=127.5.96.148:39653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 03:07:13.487875 4282 ts_manager.cc:194] Registered new tserver with Master: f0b03a5e593d4ba6a94193297706a46e (127.5.96.130:33531)
I20260812 03:07:13.488392 4282 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.96.130:50185
W20260812 03:07:13.594142 4787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:13.594365 4787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:13.594399 4787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:13.598282 4787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:13.598327 4787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:13.598377 4787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:07:13.602837 4787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:39653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:37285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:13.603732 4787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:13.604939 4787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:13.613925 4797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:13.613952 4795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:13.613930 4801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:13.614159 4787 server_base.cc:1034] running on GCE node
I20260812 03:07:13.614599 4787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:13.615159 4787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:13.616307 4787 hybrid_clock.cc:648] HybridClock initialized: now 1786504033616287 us; error 31 us; skew 500 ppm
I20260812 03:07:13.617918 4787 webserver.cc:460] Webserver started at http://127.5.96.131:33091/ using document root <none> and password file <none>
I20260812 03:07:13.618424 4787 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:13.618489 4787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:13.618680 4787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:13.620559 4787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-2/data/instance:
uuid: "98d7e1bcdbfe474284314a15fa07c7fc"
format_stamp: "Formatted at 2026-08-12 03:07:13 on dist-test-slave-6bbx"
server_key: "24bf44956ca60ee2269d57d90bde5398"
server_key_iv: "2ecc4185d49b3c01c366479f19312c97"
server_key_version: "encryptionkey@0"
I20260812 03:07:13.621064 4787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance:
uuid: "98d7e1bcdbfe474284314a15fa07c7fc"
format_stamp: "Formatted at 2026-08-12 03:07:13 on dist-test-slave-6bbx"
server_key: "24bf44956ca60ee2269d57d90bde5398"
server_key_iv: "2ecc4185d49b3c01c366479f19312c97"
server_key_version: "encryptionkey@0"
I20260812 03:07:13.624461 4787 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:07:13.626544 4810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:13.627630 4787 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:07:13.627790 4787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "98d7e1bcdbfe474284314a15fa07c7fc"
format_stamp: "Formatted at 2026-08-12 03:07:13 on dist-test-slave-6bbx"
server_key: "24bf44956ca60ee2269d57d90bde5398"
server_key_iv: "2ecc4185d49b3c01c366479f19312c97"
server_key_version: "encryptionkey@0"
I20260812 03:07:13.627880 4787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:13.643553 4787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:13.644164 4787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:13.644312 4787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:13.644820 4787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:07:13.645654 4787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:07:13.645701 4787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:13.645756 4787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:07:13.645781 4787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:13.654179 4787 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:46149
I20260812 03:07:13.654209 4979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:46149 every 8 connection(s)
I20260812 03:07:13.654970 4787 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:07:13.663175 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:13.656577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:48223 (local address 127.5.96.190:37285)
0812 03:07:13.656740 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:13.656745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:13.657743 (+ 998us) server_negotiation.cc:407] Connection header received
0812 03:07:13.659069 (+ 1326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:13.659072 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:13.659129 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:13.659203 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:13.660034 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.660576 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.661613 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.661819 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.661888 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:13.662803 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:13.662806 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:13.662809 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:13.662821 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:13.662840 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:13.662848 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:13.662906 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:13.663007 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 03:07:13.663055 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 03:07:13.663909 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 4787
I20260812 03:07:13.663937 4984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:13.656966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:37285 (local address 127.5.96.131:48223)
0812 03:07:13.657591 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:13.657629 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:13.658854 (+ 1225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:13.659353 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:13.659360 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:13.659436 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:13.659903 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:13.659916 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.660673 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.660675 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:13.661520 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:13.661529 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.661921 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.661923 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:13.662055 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:13.662569 (+ 514us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:07:13.662570 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:13.662606 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:07:13.662663 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:13.662911 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:13.662915 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:13.662918 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:07:13.662998 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 03:07:13.663186 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":114,"threads_started":1}
I20260812 03:07:13.663998 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance
I20260812 03:07:13.664273 5506 external_mini_cluster.cc:1428] Setting key 0e956ebf468c24c80cb77df321f479b2
I20260812 03:07:13.665109 4980 heartbeater.cc:347] Connected to a master server at 127.5.96.190:37285
I20260812 03:07:13.665299 4980 heartbeater.cc:464] Registering TS with master...
I20260812 03:07:13.665787 4980 heartbeater.cc:511] Master 127.5.96.190:37285 requested a full tablet report, sending...
I20260812 03:07:13.666755 4282 ts_manager.cc:194] Registered new tserver with Master: 98d7e1bcdbfe474284314a15fa07c7fc (127.5.96.131:46149)
I20260812 03:07:13.667281 4282 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.96.131:48223
I20260812 03:07:13.677665 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:07:13.683002 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:13.679439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44514 (local address 127.5.96.190:37285)
0812 03:07:13.679566 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:13.679570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:13.679743 (+ 173us) server_negotiation.cc:407] Connection header received
0812 03:07:13.679853 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:13.679856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:13.679897 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:13.679959 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:13.680221 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.680694 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.681515 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.681735 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.681819 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:13.682556 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:13.682560 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:13.682563 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:13.682575 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:13.682591 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:13.682598 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:13.682697 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:13.682839 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 03:07:13.682873 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 03:07:13.685364 4282 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44514:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:07:13.688186 4282 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:07:13.701052 5007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:13.695995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:33531 (local address 127.0.0.1:43166)
0812 03:07:13.696342 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:13.696355 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:13.696449 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:13.697431 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:13.697434 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:13.697452 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:13.697519 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:13.697525 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.698827 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.698830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:13.699657 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:13.699696 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.700556 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.700560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:13.700663 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:13.700699 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 03:07:13.700797 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 03:07:13.700911 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":2,"thread_start_us":80,"threads_started":1}
I20260812 03:07:13.701074 5009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:13.696397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43166 (local address 127.5.96.130:33531)
0812 03:07:13.697027 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:13.697041 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:13.697064 (+ 23us) server_negotiation.cc:407] Connection header received
0812 03:07:13.697134 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:13.697139 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:13.697323 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:13.697459 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:13.697636 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.698730 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.699825 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.700440 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.700539 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:13.700718 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:13.700836 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 03:07:13.700931 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":110,"threads_started":1}
I20260812 03:07:13.701334 5005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:13.695768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44110 (local address 127.5.96.131:46149)
0812 03:07:13.696020 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:13.696031 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:13.696332 (+ 301us) server_negotiation.cc:407] Connection header received
0812 03:07:13.696629 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:13.696635 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:13.696824 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:13.696967 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:13.697386 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.698244 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.699756 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.700474 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.700554 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:13.700719 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:13.701083 (+ 364us) server_negotiation.cc:299] Negotiation successful
0812 03:07:13.701181 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":88,"threads_started":1}
I20260812 03:07:13.701563 4999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:13.695318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:35319 (local address 127.0.0.1:52874)
0812 03:07:13.696196 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:13.696244 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:13.696444 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:13.697159 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:13.697161 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:13.697179 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:13.697281 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:13.697288 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.698725 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.698728 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:13.699875 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:13.699885 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.701057 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.701060 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:13.701278 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:13.701300 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:07:13.701372 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 03:07:13.701438 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"mutex_wait_us":23,"spinlock_wait_cycles":4736,"thread_start_us":75,"threads_started":1}
I20260812 03:07:13.701679 4996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:13.695451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:46149 (local address 127.0.0.1:44110)
0812 03:07:13.696196 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:13.696244 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:13.696444 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:13.697119 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:13.697124 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:13.697146 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:13.697284 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:13.697292 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.698357 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.698360 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:13.699594 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:13.699605 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.700833 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.700837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:13.700927 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:13.700950 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:07:13.701511 (+ 561us) client_negotiation.cc:240] Negotiation successful
0812 03:07:13.701572 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"mutex_wait_us":20,"spinlock_wait_cycles":7936,"thread_start_us":102,"threads_started":1}
I20260812 03:07:13.702008 5002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:13.695450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52874 (local address 127.5.96.129:35319)
0812 03:07:13.695911 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:13.695923 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:13.696334 (+ 411us) server_negotiation.cc:407] Connection header received
0812 03:07:13.696654 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:13.696659 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:13.696872 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:13.697018 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:13.697441 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.698612 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.700005 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.700744 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.700836 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:13.700999 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:13.701778 (+ 779us) server_negotiation.cc:299] Negotiation successful
0812 03:07:13.701871 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":135,"threads_started":1}
I20260812 03:07:13.704185 4475 tablet_service.cc:1467] Processing CreateTablet for tablet d261510c2d4a47bb837608694a35bdfb (DEFAULT_TABLE table=test-table [id=48a7a590ad6643cab526b899905f5e1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:07:13.704205 4687 tablet_service.cc:1467] Processing CreateTablet for tablet d261510c2d4a47bb837608694a35bdfb (DEFAULT_TABLE table=test-table [id=48a7a590ad6643cab526b899905f5e1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:07:13.704345 4879 tablet_service.cc:1467] Processing CreateTablet for tablet d261510c2d4a47bb837608694a35bdfb (DEFAULT_TABLE table=test-table [id=48a7a590ad6643cab526b899905f5e1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:07:13.705103 4475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d261510c2d4a47bb837608694a35bdfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:13.705217 4879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d261510c2d4a47bb837608694a35bdfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:13.705217 4687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d261510c2d4a47bb837608694a35bdfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:13.711180 5011 tablet_bootstrap.cc:492] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283: Bootstrap starting.
I20260812 03:07:13.712730 5013 tablet_bootstrap.cc:492] T d261510c2d4a47bb837608694a35bdfb P 98d7e1bcdbfe474284314a15fa07c7fc: Bootstrap starting.
I20260812 03:07:13.713397 5011 tablet_bootstrap.cc:654] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:13.714061 5011 log.cc:824] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:13.714427 5013 tablet_bootstrap.cc:654] T d261510c2d4a47bb837608694a35bdfb P 98d7e1bcdbfe474284314a15fa07c7fc: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:13.714670 5012 tablet_bootstrap.cc:492] T d261510c2d4a47bb837608694a35bdfb P f0b03a5e593d4ba6a94193297706a46e: Bootstrap starting.
I20260812 03:07:13.715286 5013 log.cc:824] T d261510c2d4a47bb837608694a35bdfb P 98d7e1bcdbfe474284314a15fa07c7fc: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:13.716166 5011 tablet_bootstrap.cc:492] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283: No bootstrap required, opened a new log
I20260812 03:07:13.716365 5011 ts_tablet_manager.cc:1397] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 03:07:13.716748 5012 tablet_bootstrap.cc:654] T d261510c2d4a47bb837608694a35bdfb P f0b03a5e593d4ba6a94193297706a46e: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:13.717365 5012 log.cc:824] T d261510c2d4a47bb837608694a35bdfb P f0b03a5e593d4ba6a94193297706a46e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:13.717610 5013 tablet_bootstrap.cc:492] T d261510c2d4a47bb837608694a35bdfb P 98d7e1bcdbfe474284314a15fa07c7fc: No bootstrap required, opened a new log
I20260812 03:07:13.717790 5013 ts_tablet_manager.cc:1397] T d261510c2d4a47bb837608694a35bdfb P 98d7e1bcdbfe474284314a15fa07c7fc: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 03:07:13.718878 5012 tablet_bootstrap.cc:492] T d261510c2d4a47bb837608694a35bdfb P f0b03a5e593d4ba6a94193297706a46e: No bootstrap required, opened a new log
I20260812 03:07:13.719059 5012 ts_tablet_manager.cc:1397] T d261510c2d4a47bb837608694a35bdfb P f0b03a5e593d4ba6a94193297706a46e: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260812 03:07:13.720537 5011 raft_consensus.cc:348] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d9df419e794feeae0a47309a028283" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 35319 } } peers { permanent_uuid: "98d7e1bcdbfe474284314a15fa07c7fc" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 46149 } } peers { permanent_uuid: "f0b03a5e593d4ba6a94193297706a46e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33531 } }
I20260812 03:07:13.720809 5011 raft_consensus.cc:374] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:13.720857 5011 raft_consensus.cc:729] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d9df419e794feeae0a47309a028283, State: Initialized, Role: FOLLOWER
I20260812 03:07:13.721328 5011 consensus_queue.cc:260] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d9df419e794feeae0a47309a028283" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 35319 } } peers { permanent_uuid: "98d7e1bcdbfe474284314a15fa07c7fc" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 46149 } } peers { permanent_uuid: "f0b03a5e593d4ba6a94193297706a46e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33531 } }
I20260812 03:07:13.721768 5013 raft_consensus.cc:348] T d261510c2d4a47bb837608694a35bdfb P 98d7e1bcdbfe474284314a15fa07c7fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d9df419e794feeae0a47309a028283" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 35319 } } peers { permanent_uuid: "98d7e1bcdbfe474284314a15fa07c7fc" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 46149 } } peers { permanent_uuid: "f0b03a5e593d4ba6a94193297706a46e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33531 } }
I20260812 03:07:13.721994 5013 raft_consensus.cc:374] T d261510c2d4a47bb837608694a35bdfb P 98d7e1bcdbfe474284314a15fa07c7fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:13.722020 4587 heartbeater.cc:503] Master 127.5.96.190:37285 was elected leader, sending a full tablet report...
I20260812 03:07:13.722046 5013 raft_consensus.cc:729] T d261510c2d4a47bb837608694a35bdfb P 98d7e1bcdbfe474284314a15fa07c7fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98d7e1bcdbfe474284314a15fa07c7fc, State: Initialized, Role: FOLLOWER
I20260812 03:07:13.722371 5011 ts_tablet_manager.cc:1428] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 03:07:13.722492 5013 consensus_queue.cc:260] T d261510c2d4a47bb837608694a35bdfb P 98d7e1bcdbfe474284314a15fa07c7fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d9df419e794feeae0a47309a028283" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 35319 } } peers { permanent_uuid: "98d7e1bcdbfe474284314a15fa07c7fc" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 46149 } } peers { permanent_uuid: "f0b03a5e593d4ba6a94193297706a46e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33531 } }
I20260812 03:07:13.722517 5012 raft_consensus.cc:348] T d261510c2d4a47bb837608694a35bdfb P f0b03a5e593d4ba6a94193297706a46e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d9df419e794feeae0a47309a028283" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 35319 } } peers { permanent_uuid: "98d7e1bcdbfe474284314a15fa07c7fc" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 46149 } } peers { permanent_uuid: "f0b03a5e593d4ba6a94193297706a46e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33531 } }
I20260812 03:07:13.722735 5012 raft_consensus.cc:374] T d261510c2d4a47bb837608694a35bdfb P f0b03a5e593d4ba6a94193297706a46e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:13.722793 5012 raft_consensus.cc:729] T d261510c2d4a47bb837608694a35bdfb P f0b03a5e593d4ba6a94193297706a46e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0b03a5e593d4ba6a94193297706a46e, State: Initialized, Role: FOLLOWER
I20260812 03:07:13.723163 5012 consensus_queue.cc:260] T d261510c2d4a47bb837608694a35bdfb P f0b03a5e593d4ba6a94193297706a46e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d9df419e794feeae0a47309a028283" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 35319 } } peers { permanent_uuid: "98d7e1bcdbfe474284314a15fa07c7fc" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 46149 } } peers { permanent_uuid: "f0b03a5e593d4ba6a94193297706a46e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33531 } }
I20260812 03:07:13.723371 4980 heartbeater.cc:503] Master 127.5.96.190:37285 was elected leader, sending a full tablet report...
I20260812 03:07:13.723448 5013 ts_tablet_manager.cc:1428] T d261510c2d4a47bb837608694a35bdfb P 98d7e1bcdbfe474284314a15fa07c7fc: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 03:07:13.723939 4784 heartbeater.cc:503] Master 127.5.96.190:37285 was elected leader, sending a full tablet report...
I20260812 03:07:13.724063 5012 ts_tablet_manager.cc:1428] T d261510c2d4a47bb837608694a35bdfb P f0b03a5e593d4ba6a94193297706a46e: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
W20260812 03:07:13.726737 4785 tablet.cc:2369] T d261510c2d4a47bb837608694a35bdfb P f0b03a5e593d4ba6a94193297706a46e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:07:13.731179 5019 raft_consensus.cc:482] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:07:13.731297 5019 raft_consensus.cc:504] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d9df419e794feeae0a47309a028283" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 35319 } } peers { permanent_uuid: "98d7e1bcdbfe474284314a15fa07c7fc" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 46149 } } peers { permanent_uuid: "f0b03a5e593d4ba6a94193297706a46e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33531 } }
I20260812 03:07:13.732275 5019 leader_election.cc:288] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98d7e1bcdbfe474284314a15fa07c7fc (127.5.96.131:46149), f0b03a5e593d4ba6a94193297706a46e (127.5.96.130:33531)
I20260812 03:07:13.735728 5005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:13.732635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:36151 (local address 127.5.96.131:46149)
0812 03:07:13.732737 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:13.732742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:13.732756 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:07:13.732795 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:13.732798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:13.732840 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:13.732913 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:13.733204 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.733728 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.734795 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.735161 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.735260 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:13.735307 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:13.735531 (+ 224us) server_negotiation.cc:299] Negotiation successful
0812 03:07:13.735590 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:07:13.735729 4589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:13.732417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:46149 (local address 127.5.96.129:36151)
0812 03:07:13.732524 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:13.732540 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:13.732618 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:13.732933 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:13.732936 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:13.732951 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:13.733073 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:13.733081 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.733831 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.733834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:13.734672 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:13.734683 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.735261 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.735264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:13.735401 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:13.735447 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 03:07:13.735515 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 03:07:13.735584 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":31}
I20260812 03:07:13.735802 5009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:13.732650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:57061 (local address 127.5.96.130:33531)
0812 03:07:13.732742 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:13.732746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:13.732827 (+ 81us) server_negotiation.cc:407] Connection header received
0812 03:07:13.732901 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:13.732904 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:13.732951 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:13.733015 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:13.733275 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.733744 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.734800 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.735175 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.735275 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:13.735338 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:13.735592 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 03:07:13.735692 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 03:07:13.736126 5023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:13.732485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:33531 (local address 127.5.96.129:57061)
0812 03:07:13.732762 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:13.732772 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:13.732840 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:13.733030 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:13.733033 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:13.733046 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:13.733177 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:13.733183 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.733851 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.733854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:13.734668 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:13.734679 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.735282 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.735286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:13.735447 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:13.735468 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 03:07:13.735917 (+ 449us) client_negotiation.cc:240] Negotiation successful
0812 03:07:13.735987 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":74,"thread_start_us":77,"threads_started":1}
I20260812 03:07:13.736254 4908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d261510c2d4a47bb837608694a35bdfb" candidate_uuid: "c4d9df419e794feeae0a47309a028283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98d7e1bcdbfe474284314a15fa07c7fc" is_pre_election: true
I20260812 03:07:13.736491 4908 raft_consensus.cc:2454] T d261510c2d4a47bb837608694a35bdfb P 98d7e1bcdbfe474284314a15fa07c7fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d9df419e794feeae0a47309a028283 in term 0.
I20260812 03:07:13.736600 4716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d261510c2d4a47bb837608694a35bdfb" candidate_uuid: "c4d9df419e794feeae0a47309a028283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b03a5e593d4ba6a94193297706a46e" is_pre_election: true
I20260812 03:07:13.736807 4716 raft_consensus.cc:2454] T d261510c2d4a47bb837608694a35bdfb P f0b03a5e593d4ba6a94193297706a46e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d9df419e794feeae0a47309a028283 in term 0.
I20260812 03:07:13.736999 4393 leader_election.cc:302] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98d7e1bcdbfe474284314a15fa07c7fc, c4d9df419e794feeae0a47309a028283; no voters:
I20260812 03:07:13.737246 5019 raft_consensus.cc:2781] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:07:13.737318 5019 raft_consensus.cc:482] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:07:13.737368 5019 raft_consensus.cc:3037] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:13.738315 5019 raft_consensus.cc:504] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d9df419e794feeae0a47309a028283" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 35319 } } peers { permanent_uuid: "98d7e1bcdbfe474284314a15fa07c7fc" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 46149 } } peers { permanent_uuid: "f0b03a5e593d4ba6a94193297706a46e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33531 } }
I20260812 03:07:13.738703 5019 leader_election.cc:288] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283 [CANDIDATE]: Term 1 election: Requested vote from peers 98d7e1bcdbfe474284314a15fa07c7fc (127.5.96.131:46149), f0b03a5e593d4ba6a94193297706a46e (127.5.96.130:33531)
I20260812 03:07:13.739061 4716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d261510c2d4a47bb837608694a35bdfb" candidate_uuid: "c4d9df419e794feeae0a47309a028283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b03a5e593d4ba6a94193297706a46e"
I20260812 03:07:13.739192 4716 raft_consensus.cc:3037] T d261510c2d4a47bb837608694a35bdfb P f0b03a5e593d4ba6a94193297706a46e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:13.739779 4908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d261510c2d4a47bb837608694a35bdfb" candidate_uuid: "c4d9df419e794feeae0a47309a028283" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98d7e1bcdbfe474284314a15fa07c7fc"
I20260812 03:07:13.739882 4908 raft_consensus.cc:3037] T d261510c2d4a47bb837608694a35bdfb P 98d7e1bcdbfe474284314a15fa07c7fc [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:13.740185 4716 raft_consensus.cc:2454] T d261510c2d4a47bb837608694a35bdfb P f0b03a5e593d4ba6a94193297706a46e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d9df419e794feeae0a47309a028283 in term 1.
I20260812 03:07:13.740461 4398 leader_election.cc:302] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4d9df419e794feeae0a47309a028283, f0b03a5e593d4ba6a94193297706a46e; no voters:
I20260812 03:07:13.740630 5019 raft_consensus.cc:2781] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:13.740847 5019 raft_consensus.cc:686] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283 [term 1 LEADER]: Becoming Leader. State: Replica: c4d9df419e794feeae0a47309a028283, State: Running, Role: LEADER
I20260812 03:07:13.741041 4908 raft_consensus.cc:2454] T d261510c2d4a47bb837608694a35bdfb P 98d7e1bcdbfe474284314a15fa07c7fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d9df419e794feeae0a47309a028283 in term 1.
I20260812 03:07:13.741165 5019 consensus_queue.cc:237] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d9df419e794feeae0a47309a028283" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 35319 } } peers { permanent_uuid: "98d7e1bcdbfe474284314a15fa07c7fc" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 46149 } } peers { permanent_uuid: "f0b03a5e593d4ba6a94193297706a46e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33531 } }
I20260812 03:07:13.744066 4280 catalog_manager.cc:5495] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4d9df419e794feeae0a47309a028283 (127.5.96.129). New cstate: current_term: 1 leader_uuid: "c4d9df419e794feeae0a47309a028283" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d9df419e794feeae0a47309a028283" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 35319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98d7e1bcdbfe474284314a15fa07c7fc" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 46149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0b03a5e593d4ba6a94193297706a46e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33531 } health_report { overall_health: UNKNOWN } } }
I20260812 03:07:13.753639 5002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:13.750485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52876 (local address 127.5.96.129:35319)
0812 03:07:13.750585 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:13.750588 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:13.750600 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:07:13.750753 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:13.750756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:13.750806 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:13.750883 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:07:13.751062 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.751563 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.752427 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:13.752624 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:13.752690 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:13.752907 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:07:13.753232 (+ 325us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:07:13.753341 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:13.753482 (+ 141us) server_negotiation.cc:299] Negotiation successful
0812 03:07:13.753511 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 03:07:13.758644 4716 raft_consensus.cc:1261] T d261510c2d4a47bb837608694a35bdfb P f0b03a5e593d4ba6a94193297706a46e [term 1 FOLLOWER]: Refusing update from remote peer c4d9df419e794feeae0a47309a028283: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:07:13.758646 4908 raft_consensus.cc:1261] T d261510c2d4a47bb837608694a35bdfb P 98d7e1bcdbfe474284314a15fa07c7fc [term 1 FOLLOWER]: Refusing update from remote peer c4d9df419e794feeae0a47309a028283: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:07:13.759233 5019 consensus_queue.cc:1035] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98d7e1bcdbfe474284314a15fa07c7fc" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 46149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:07:13.759393 5024 consensus_queue.cc:1035] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0b03a5e593d4ba6a94193297706a46e" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 33531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:07:13.766881 5032 mvcc.cc:205] Tried to move back new op lower bound from 7317520522269315072 to 7317520522206162944. Current Snapshot: MvccSnapshot[applied={T|T < 7317520522269315072}]
I20260812 03:07:13.773434 4280 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:07:13.773557 4280 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:07:13.775760 4280 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d7b37fab0d4427282f56b4ebff71a26: Sending DeleteTablet for 3 replicas of tablet d261510c2d4a47bb837608694a35bdfb
I20260812 03:07:13.776510 4879 tablet_service.cc:1514] Processing DeleteTablet for tablet d261510c2d4a47bb837608694a35bdfb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:07:13 UTC) from {username='slave'} at 127.0.0.1:44110
I20260812 03:07:13.776701 4475 tablet_service.cc:1514] Processing DeleteTablet for tablet d261510c2d4a47bb837608694a35bdfb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:07:13 UTC) from {username='slave'} at 127.0.0.1:52874
I20260812 03:07:13.776705 4687 tablet_service.cc:1514] Processing DeleteTablet for tablet d261510c2d4a47bb837608694a35bdfb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:07:13 UTC) from {username='slave'} at 127.0.0.1:43166
I20260812 03:07:13.777096 5045 tablet_replica.cc:330] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283: stopping tablet replica
I20260812 03:07:13.777390 5045 raft_consensus.cc:2229] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:07:13.777438 5044 tablet_replica.cc:330] T d261510c2d4a47bb837608694a35bdfb P 98d7e1bcdbfe474284314a15fa07c7fc: stopping tablet replica
I20260812 03:07:13.777678 5044 raft_consensus.cc:2229] T d261510c2d4a47bb837608694a35bdfb P 98d7e1bcdbfe474284314a15fa07c7fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:07:13.777755 5045 raft_consensus.cc:2258] T d261510c2d4a47bb837608694a35bdfb P c4d9df419e794feeae0a47309a028283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:07:13.777758 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 4372
I20260812 03:07:13.778473 5048 tablet_replica.cc:330] T d261510c2d4a47bb837608694a35bdfb P f0b03a5e593d4ba6a94193297706a46e: stopping tablet replica
I20260812 03:07:13.778704 5048 raft_consensus.cc:2229] T d261510c2d4a47bb837608694a35bdfb P f0b03a5e593d4ba6a94193297706a46e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:07:13.778905 5048 raft_consensus.cc:2258] T d261510c2d4a47bb837608694a35bdfb P f0b03a5e593d4ba6a94193297706a46e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:07:13.777952 5044 raft_consensus.cc:2258] T d261510c2d4a47bb837608694a35bdfb P 98d7e1bcdbfe474284314a15fa07c7fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:07:13.787041 5048 ts_tablet_manager.cc:1905] T d261510c2d4a47bb837608694a35bdfb P f0b03a5e593d4ba6a94193297706a46e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:07:13.789178 5048 ts_tablet_manager.cc:1918] T d261510c2d4a47bb837608694a35bdfb P f0b03a5e593d4ba6a94193297706a46e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:07:13.789268 5048 log.cc:1196] T d261510c2d4a47bb837608694a35bdfb P f0b03a5e593d4ba6a94193297706a46e: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-1/wal/wals/d261510c2d4a47bb837608694a35bdfb
I20260812 03:07:13.789518 5048 ts_tablet_manager.cc:1939] T d261510c2d4a47bb837608694a35bdfb P f0b03a5e593d4ba6a94193297706a46e: Deleting consensus metadata
I20260812 03:07:13.790529 4261 catalog_manager.cc:4841] TS f0b03a5e593d4ba6a94193297706a46e (127.5.96.130:33531): tablet d261510c2d4a47bb837608694a35bdfb (table test-table [id=48a7a590ad6643cab526b899905f5e1b]) successfully deleted
I20260812 03:07:13.791407 5044 ts_tablet_manager.cc:1905] T d261510c2d4a47bb837608694a35bdfb P 98d7e1bcdbfe474284314a15fa07c7fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:07:13.793855 5044 ts_tablet_manager.cc:1918] T d261510c2d4a47bb837608694a35bdfb P 98d7e1bcdbfe474284314a15fa07c7fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:07:13.793954 5044 log.cc:1196] T d261510c2d4a47bb837608694a35bdfb P 98d7e1bcdbfe474284314a15fa07c7fc: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786503790637001-5506-0/minicluster-data/ts-2/wal/wals/d261510c2d4a47bb837608694a35bdfb
I20260812 03:07:13.794220 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 4591
W20260812 03:07:13.800302 4261 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.96.129:35319 (error 108)
I20260812 03:07:13.810055 4999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:13.809738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:35319 (local address 127.0.0.1:52888)
0812 03:07:13.809914 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:13.809972 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.129:35319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260812 03:07:13.810242 4261 catalog_manager.cc:4584] TS c4d9df419e794feeae0a47309a028283 (127.5.96.129:35319): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d261510c2d4a47bb837608694a35bdfb: Network error: Client connection negotiation failed: client connection to 127.5.96.129:35319: connect: Connection refused (error 111)
I20260812 03:07:13.794271 5044 ts_tablet_manager.cc:1939] T d261510c2d4a47bb837608694a35bdfb P 98d7e1bcdbfe474284314a15fa07c7fc: Deleting consensus metadata
I20260812 03:07:13.837654 4256 catalog_manager.cc:4841] TS 98d7e1bcdbfe474284314a15fa07c7fc (127.5.96.131:46149): tablet d261510c2d4a47bb837608694a35bdfb (table test-table [id=48a7a590ad6643cab526b899905f5e1b]) successfully deleted
I20260812 03:07:13.841952 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 4787
I20260812 03:07:13.853962 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 4234
2026-08-12T03:07:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1081 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:07:13 dist-test-slave-6bbx krb5kdc[5053](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:07:13 dist-test-slave-6bbx krb5kdc[5053](info): set up 2 sockets
Aug 12 03:07:13 dist-test-slave-6bbx krb5kdc[5053](info): commencing operation
krb5kdc: starting...
W20260812 03:07:15.014887 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:07:15 dist-test-slave-6bbx krb5kdc[5053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:07:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:07:15.125811 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:45103
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:37485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:45103
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:15.231878 5075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:15.232095 5075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:15.232131 5075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:15.235889 5075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:15.235929 5075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:15.235946 5075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:15.235961 5075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:15.235975 5075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:15.240866 5075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:37485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:45103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:45103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:15.241688 5075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:15.242573 5075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:15.250298 5085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:15.250362 5084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:15.250308 5088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:15.250532 5075 server_base.cc:1034] running on GCE node
I20260812 03:07:15.251113 5075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:15.252130 5075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:15.253289 5075 hybrid_clock.cc:648] HybridClock initialized: now 1786504035253271 us; error 30 us; skew 500 ppm
Aug 12 03:07:15 dist-test-slave-6bbx krb5kdc[5053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504035, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:15.255877 5075 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:07:15.256685 5075 webserver.cc:460] Webserver started at http://127.5.96.190:39107/ using document root <none> and password file <none>
I20260812 03:07:15.257177 5075 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:15.257242 5075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:15.257422 5075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:15.259343 5075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "79c7343281784981af3c2a1ea03102ca"
format_stamp: "Formatted at 2026-08-12 03:07:15 on dist-test-slave-6bbx"
server_key: "bf8954abf73a45118da26d55ee088757"
server_key_iv: "7d1beb4f377bdb6364bde485197cc51b"
server_key_version: "encryptionkey@0"
I20260812 03:07:15.259899 5075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "79c7343281784981af3c2a1ea03102ca"
format_stamp: "Formatted at 2026-08-12 03:07:15 on dist-test-slave-6bbx"
server_key: "bf8954abf73a45118da26d55ee088757"
server_key_iv: "7d1beb4f377bdb6364bde485197cc51b"
server_key_version: "encryptionkey@0"
I20260812 03:07:15.263316 5075 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:07:15.265528 5102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:15.266577 5075 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:07:15.266685 5075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "79c7343281784981af3c2a1ea03102ca"
format_stamp: "Formatted at 2026-08-12 03:07:15 on dist-test-slave-6bbx"
server_key: "bf8954abf73a45118da26d55ee088757"
server_key_iv: "7d1beb4f377bdb6364bde485197cc51b"
server_key_version: "encryptionkey@0"
I20260812 03:07:15.266780 5075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:15.297415 5075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:15.299849 5075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:15.300009 5075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:15.306511 5075 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:45103
I20260812 03:07:15.306524 5176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:45103 every 8 connection(s)
I20260812 03:07:15.307358 5075 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:15.309988 5177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:15.311126 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 5075
I20260812 03:07:15.311257 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:07:15.311519 5506 external_mini_cluster.cc:1428] Setting key 95a37e81dd106f3ba788477fc422ad7d
I20260812 03:07:15.316196 5177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79c7343281784981af3c2a1ea03102ca: Bootstrap starting.
I20260812 03:07:15.318650 5177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79c7343281784981af3c2a1ea03102ca: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:15.319412 5177 log.cc:824] T 00000000000000000000000000000000 P 79c7343281784981af3c2a1ea03102ca: Log is configured to *not* fsync() on all Append() calls
Aug 12 03:07:15 dist-test-slave-6bbx krb5kdc[5053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:07:15.321276 5177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79c7343281784981af3c2a1ea03102ca: No bootstrap required, opened a new log
I20260812 03:07:15.325616 5183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:15.312809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56706 (local address 127.5.96.190:45103)
0812 03:07:15.313213 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:15.313232 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:15.313273 (+ 41us) server_negotiation.cc:407] Connection header received
0812 03:07:15.314063 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:15.314086 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:15.314480 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:15.314824 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:15.315456 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:15.316851 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:15.317668 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:15.318321 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:15.318470 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:15.320793 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:15.320800 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:15.320814 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:15.320843 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:15.322635 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:15.323028 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:15.323034 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:15.323039 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:15.323180 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:15.323397 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:15.323400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:15.323402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:15.323974 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:15.324158 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:15.324421 (+ 263us) server_negotiation.cc:299] Negotiation successful
0812 03:07:15.324707 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":112,"threads_started":1}
I20260812 03:07:15.325627 5177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79c7343281784981af3c2a1ea03102ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79c7343281784981af3c2a1ea03102ca" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45103 } }
I20260812 03:07:15.325922 5177 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79c7343281784981af3c2a1ea03102ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:15.325968 5177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79c7343281784981af3c2a1ea03102ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c7343281784981af3c2a1ea03102ca, State: Initialized, Role: FOLLOWER
I20260812 03:07:15.326354 5177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79c7343281784981af3c2a1ea03102ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79c7343281784981af3c2a1ea03102ca" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45103 } }
I20260812 03:07:15.326464 5177 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79c7343281784981af3c2a1ea03102ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:07:15.326502 5177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79c7343281784981af3c2a1ea03102ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:07:15.326572 5177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79c7343281784981af3c2a1ea03102ca [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:15.327580 5177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79c7343281784981af3c2a1ea03102ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79c7343281784981af3c2a1ea03102ca" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45103 } }
I20260812 03:07:15.327951 5177 leader_election.cc:302] T 00000000000000000000000000000000 P 79c7343281784981af3c2a1ea03102ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79c7343281784981af3c2a1ea03102ca; no voters:
I20260812 03:07:15.328234 5177 leader_election.cc:288] T 00000000000000000000000000000000 P 79c7343281784981af3c2a1ea03102ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:07:15.328325 5185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79c7343281784981af3c2a1ea03102ca [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:15.328522 5185 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79c7343281784981af3c2a1ea03102ca [term 1 LEADER]: Becoming Leader. State: Replica: 79c7343281784981af3c2a1ea03102ca, State: Running, Role: LEADER
I20260812 03:07:15.328812 5185 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79c7343281784981af3c2a1ea03102ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79c7343281784981af3c2a1ea03102ca" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45103 } }
I20260812 03:07:15.329312 5177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79c7343281784981af3c2a1ea03102ca [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:15.330345 5187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79c7343281784981af3c2a1ea03102ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79c7343281784981af3c2a1ea03102ca. Latest consensus state: current_term: 1 leader_uuid: "79c7343281784981af3c2a1ea03102ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79c7343281784981af3c2a1ea03102ca" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45103 } } }
I20260812 03:07:15.330471 5187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79c7343281784981af3c2a1ea03102ca [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:15.330729 5186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79c7343281784981af3c2a1ea03102ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79c7343281784981af3c2a1ea03102ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79c7343281784981af3c2a1ea03102ca" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45103 } } }
I20260812 03:07:15.330819 5197 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:15.330838 5186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79c7343281784981af3c2a1ea03102ca [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:15.333858 5197 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:15.339200 5197 catalog_manager.cc:1347] Generated new cluster ID: 607df28ecaca4fbc9a5f1469e57987b8
I20260812 03:07:15.339260 5197 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:15.352366 5197 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:15.353127 5197 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:15.368990 5197 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79c7343281784981af3c2a1ea03102ca: Generated new TSK 0
I20260812 03:07:15.369597 5197 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.96.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
Entry for principal kudu/127.5.96.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab.
I20260812 03:07:15.403885 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:0
--local_ip_for_outbound_sockets=127.5.96.129
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:45103
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:37485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:15.511063 5220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:15.511279 5220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:15.511317 5220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:15.515144 5220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:15.515192 5220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:15.515244 5220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:07:15.520228 5220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:37485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.129.keytab
--principal=kudu/127.5.96.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:45103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:15.521068 5220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:15.521943 5220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:15.530869 5225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:15.530876 5226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:15.530894 5229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:15.531069 5220 server_base.cc:1034] running on GCE node
I20260812 03:07:15.531433 5220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:15.532080 5220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:15.533239 5220 hybrid_clock.cc:648] HybridClock initialized: now 1786504035533225 us; error 28 us; skew 500 ppm
Aug 12 03:07:15 dist-test-slave-6bbx krb5kdc[5053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504035, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:15.535821 5220 init.cc:377] Logged in from keytab as kudu/127.5.96.129@KRBTEST.COM (short username kudu)
I20260812 03:07:15.536602 5220 webserver.cc:460] Webserver started at http://127.5.96.129:44491/ using document root <none> and password file <none>
I20260812 03:07:15.537097 5220 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:15.537155 5220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:15.537343 5220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:15.539254 5220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-0/data/instance:
uuid: "c738b378e4b6448b95ecb3f444a037d5"
format_stamp: "Formatted at 2026-08-12 03:07:15 on dist-test-slave-6bbx"
server_key: "4c8178b6642505035c419aff607aec61"
server_key_iv: "b3d47811418525776814f624a90af6f9"
server_key_version: "encryptionkey@0"
I20260812 03:07:15.539793 5220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance:
uuid: "c738b378e4b6448b95ecb3f444a037d5"
format_stamp: "Formatted at 2026-08-12 03:07:15 on dist-test-slave-6bbx"
server_key: "4c8178b6642505035c419aff607aec61"
server_key_iv: "b3d47811418525776814f624a90af6f9"
server_key_version: "encryptionkey@0"
I20260812 03:07:15.543113 5220 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:07:15.545289 5242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:15.546343 5220 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260812 03:07:15.546447 5220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "c738b378e4b6448b95ecb3f444a037d5"
format_stamp: "Formatted at 2026-08-12 03:07:15 on dist-test-slave-6bbx"
server_key: "4c8178b6642505035c419aff607aec61"
server_key_iv: "b3d47811418525776814f624a90af6f9"
server_key_version: "encryptionkey@0"
I20260812 03:07:15.546537 5220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:15.592006 5220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:15.594873 5220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:15.595036 5220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:15.595575 5220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:07:15.596362 5220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:07:15.596408 5220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:15.596449 5220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:07:15.596477 5220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:15.604632 5220 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:35345
I20260812 03:07:15.604667 5418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:35345 every 8 connection(s)
I20260812 03:07:15.605436 5220 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:07:15.609669 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 5220
I20260812 03:07:15.609792 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance
I20260812 03:07:15.610127 5506 external_mini_cluster.cc:1428] Setting key 66ab529c4e0f2f29766bb0d54a50c64b
Aug 12 03:07:15 dist-test-slave-6bbx krb5kdc[5053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504035, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.129@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:07:15.618338 5183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:15.607050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:52179 (local address 127.5.96.190:45103)
0812 03:07:15.607168 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:15.607171 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:15.607948 (+ 777us) server_negotiation.cc:407] Connection header received
0812 03:07:15.609176 (+ 1228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:15.609178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:15.609223 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:15.609296 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:15.610586 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:15.611186 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:15.612385 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:15.612609 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:15.612695 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:15.614920 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:15.614924 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:15.614927 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:15.614954 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:15.616477 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:15.616935 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:15.616938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:15.616940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:15.617037 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:15.617328 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:15.617332 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:15.617333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:15.617550 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:15.617649 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:15.618057 (+ 408us) server_negotiation.cc:299] Negotiation successful
0812 03:07:15.618177 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 03:07:15.619036 5422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:15.607378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:45103 (local address 127.5.96.129:52179)
0812 03:07:15.607817 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:15.607858 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:15.608999 (+ 1141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:15.609434 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:15.609440 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:15.609892 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:15.610426 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:15.610441 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:15.611320 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:15.611323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:15.612260 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:15.612271 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:15.612742 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:15.612746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:15.612903 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:15.613415 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:15.613430 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:15.614764 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:15.616584 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:15.616589 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:15.616600 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:15.616839 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:15.617119 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:15.617121 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:15.617123 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:15.617239 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:15.617641 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:15.617646 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:15.617843 (+ 197us) client_negotiation.cc:769] Sending connection context
0812 03:07:15.617980 (+ 137us) client_negotiation.cc:240] Negotiation successful
0812 03:07:15.618193 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":107,"threads_started":1}
WARNING: no policy specified for kudu/127.5.96.130@KRBTEST.COM; defaulting to no policy
I20260812 03:07:15.620179 5419 heartbeater.cc:347] Connected to a master server at 127.5.96.190:45103
I20260812 03:07:15.620379 5419 heartbeater.cc:464] Registering TS with master...
I20260812 03:07:15.620797 5419 heartbeater.cc:511] Master 127.5.96.190:45103 requested a full tablet report, sending...
I20260812 03:07:15.622151 5127 ts_manager.cc:194] Registered new tserver with Master: c738b378e4b6448b95ecb3f444a037d5 (127.5.96.129:35345)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.130@KRBTEST.COM" created.
I20260812 03:07:15.623296 5127 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.129@KRBTEST.COM'} at 127.5.96.129:52179
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
Entry for principal kudu/127.5.96.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab.
I20260812 03:07:15.634956 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:0
--local_ip_for_outbound_sockets=127.5.96.130
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:45103
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:37485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:15.741827 5427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:15.742035 5427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:15.742072 5427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:15.745882 5427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:15.745926 5427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:15.745980 5427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:07:15.750871 5427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:37485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.130.keytab
--principal=kudu/127.5.96.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:45103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:15.751744 5427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:15.752610 5427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:15.761467 5436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:15.761536 5438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:15.761619 5427 server_base.cc:1034] running on GCE node
W20260812 03:07:15.761536 5435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:15.762063 5427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:15.762640 5427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:15.763792 5427 hybrid_clock.cc:648] HybridClock initialized: now 1786504035763771 us; error 26 us; skew 500 ppm
Aug 12 03:07:15 dist-test-slave-6bbx krb5kdc[5053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504035, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:15.766409 5427 init.cc:377] Logged in from keytab as kudu/127.5.96.130@KRBTEST.COM (short username kudu)
I20260812 03:07:15.767153 5427 webserver.cc:460] Webserver started at http://127.5.96.130:42167/ using document root <none> and password file <none>
I20260812 03:07:15.767640 5427 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:15.767740 5427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:15.767933 5427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:15.769848 5427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-1/data/instance:
uuid: "f6bb443eacc545b9b8edd2b100457f53"
format_stamp: "Formatted at 2026-08-12 03:07:15 on dist-test-slave-6bbx"
server_key: "65902a45305daa78d4cfd9763df084fd"
server_key_iv: "799d198861b0c66a218579fe127f31c5"
server_key_version: "encryptionkey@0"
I20260812 03:07:15.770370 5427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance:
uuid: "f6bb443eacc545b9b8edd2b100457f53"
format_stamp: "Formatted at 2026-08-12 03:07:15 on dist-test-slave-6bbx"
server_key: "65902a45305daa78d4cfd9763df084fd"
server_key_iv: "799d198861b0c66a218579fe127f31c5"
server_key_version: "encryptionkey@0"
I20260812 03:07:15.773717 5427 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 03:07:15.775875 5451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:15.776907 5427 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:07:15.777009 5427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "f6bb443eacc545b9b8edd2b100457f53"
format_stamp: "Formatted at 2026-08-12 03:07:15 on dist-test-slave-6bbx"
server_key: "65902a45305daa78d4cfd9763df084fd"
server_key_iv: "799d198861b0c66a218579fe127f31c5"
server_key_version: "encryptionkey@0"
I20260812 03:07:15.777092 5427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:15.786356 5427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:15.789086 5427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:15.789237 5427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:15.789746 5427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:07:15.790473 5427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:07:15.790519 5427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:15.790568 5427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:07:15.790594 5427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:15.798476 5427 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:45095
I20260812 03:07:15.798533 5627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:45095 every 8 connection(s)
I20260812 03:07:15.799223 5427 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:07:15.799811 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 5427
I20260812 03:07:15.799935 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance
I20260812 03:07:15.800230 5506 external_mini_cluster.cc:1428] Setting key 4fba006f1a778052fee5f35c17daaed7
Aug 12 03:07:15 dist-test-slave-6bbx krb5kdc[5053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504035, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.130@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.96.131@KRBTEST.COM; defaulting to no policy
I20260812 03:07:15.813192 5183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:15.801100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:47343 (local address 127.5.96.190:45103)
0812 03:07:15.801515 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:15.801519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:15.802317 (+ 798us) server_negotiation.cc:407] Connection header received
0812 03:07:15.804031 (+ 1714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:15.804034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:15.804089 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:15.804176 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:15.805457 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:15.805982 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:15.807110 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:15.807345 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:15.807436 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:15.809708 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:15.809712 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:15.809714 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:15.809742 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:15.811106 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:15.811600 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:15.811604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:15.811606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:15.811728 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:15.812036 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:15.812039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:15.812040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:15.812278 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:15.812374 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:15.812874 (+ 500us) server_negotiation.cc:299] Negotiation successful
0812 03:07:15.813014 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.131@KRBTEST.COM" created.
I20260812 03:07:15.814033 5630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:15.801493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:45103 (local address 127.5.96.130:47343)
0812 03:07:15.802142 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:15.802224 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:15.803812 (+ 1588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:15.804310 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:15.804316 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:15.804681 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:15.805307 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:15.805326 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:15.806103 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:15.806107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:15.806991 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:15.807001 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:15.807485 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:15.807489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:15.807646 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:15.808185 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:15.808201 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:15.809552 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:15.811213 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:15.811219 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:15.811232 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:15.811475 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:15.811813 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:15.811815 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:15.811816 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:15.811941 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:15.812389 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:15.812396 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:15.812643 (+ 247us) client_negotiation.cc:769] Sending connection context
0812 03:07:15.812812 (+ 169us) client_negotiation.cc:240] Negotiation successful
0812 03:07:15.813115 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":153,"threads_started":1}
I20260812 03:07:15.815299 5628 heartbeater.cc:347] Connected to a master server at 127.5.96.190:45103
I20260812 03:07:15.815517 5628 heartbeater.cc:464] Registering TS with master...
I20260812 03:07:15.816102 5628 heartbeater.cc:511] Master 127.5.96.190:45103 requested a full tablet report, sending...
I20260812 03:07:15.817245 5127 ts_manager.cc:194] Registered new tserver with Master: f6bb443eacc545b9b8edd2b100457f53 (127.5.96.130:45095)
I20260812 03:07:15.817795 5127 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.130@KRBTEST.COM'} at 127.5.96.130:47343
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
Entry for principal kudu/127.5.96.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab.
I20260812 03:07:15.828008 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:0
--local_ip_for_outbound_sockets=127.5.96.131
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:45103
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:37485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:15.935099 5635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:15.935322 5635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:15.935366 5635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:15.939254 5635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:15.939301 5635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:15.939365 5635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:07:15.944325 5635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:37485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.131.keytab
--principal=kudu/127.5.96.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:45103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:15.945205 5635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:15.946106 5635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:15.954787 5645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:15.954857 5644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:15.954952 5635 server_base.cc:1034] running on GCE node
W20260812 03:07:15.954854 5647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:15.955380 5635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:15.955953 5635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:15.957106 5635 hybrid_clock.cc:648] HybridClock initialized: now 1786504035957084 us; error 32 us; skew 500 ppm
Aug 12 03:07:15 dist-test-slave-6bbx krb5kdc[5053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504035, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:15.959640 5635 init.cc:377] Logged in from keytab as kudu/127.5.96.131@KRBTEST.COM (short username kudu)
I20260812 03:07:15.960417 5635 webserver.cc:460] Webserver started at http://127.5.96.131:46557/ using document root <none> and password file <none>
I20260812 03:07:15.960897 5635 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:15.960955 5635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:15.961127 5635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:15.962998 5635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-2/data/instance:
uuid: "27101ddbad354f99bc0cf3dc2464974b"
format_stamp: "Formatted at 2026-08-12 03:07:15 on dist-test-slave-6bbx"
server_key: "9a888da6df93e9d5ef4050f294ad9681"
server_key_iv: "396e1f3a77ed07b70aeb9e5e542bd728"
server_key_version: "encryptionkey@0"
I20260812 03:07:15.963516 5635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance:
uuid: "27101ddbad354f99bc0cf3dc2464974b"
format_stamp: "Formatted at 2026-08-12 03:07:15 on dist-test-slave-6bbx"
server_key: "9a888da6df93e9d5ef4050f294ad9681"
server_key_iv: "396e1f3a77ed07b70aeb9e5e542bd728"
server_key_version: "encryptionkey@0"
I20260812 03:07:15.966845 5635 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:07:15.969027 5660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:15.970105 5635 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:07:15.970198 5635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "27101ddbad354f99bc0cf3dc2464974b"
format_stamp: "Formatted at 2026-08-12 03:07:15 on dist-test-slave-6bbx"
server_key: "9a888da6df93e9d5ef4050f294ad9681"
server_key_iv: "396e1f3a77ed07b70aeb9e5e542bd728"
server_key_version: "encryptionkey@0"
I20260812 03:07:15.970297 5635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:15.993789 5635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:15.996268 5635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:15.996420 5635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:15.996934 5635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:07:15.997677 5635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:07:15.997722 5635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:15.997761 5635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:07:15.997790 5635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:16.005707 5635 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:32779
I20260812 03:07:16.005738 5829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:32779 every 8 connection(s)
I20260812 03:07:16.006510 5635 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:07:16.013669 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 5635
I20260812 03:07:16.013782 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance
I20260812 03:07:16.014101 5506 external_mini_cluster.cc:1428] Setting key b0a2a78cf5b9c3ffc56a7ad8be87bcab
Aug 12 03:07:16 dist-test-slave-6bbx krb5kdc[5053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504035, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.131@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:07:16.018838 5183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:16.008143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:49505 (local address 127.5.96.190:45103)
0812 03:07:16.008270 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:16.008274 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:16.009053 (+ 779us) server_negotiation.cc:407] Connection header received
0812 03:07:16.010264 (+ 1211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:16.010268 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:16.010315 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:16.010383 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:16.011502 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.012047 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.013087 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.013289 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.013354 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:16.015887 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:16.015890 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:16.015892 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:16.015918 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:16.017227 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:16.017669 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:16.017671 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:16.017673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:16.017741 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:16.017997 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:16.017999 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:16.018000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:16.018156 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:16.018223 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:16.018616 (+ 393us) server_negotiation.cc:299] Negotiation successful
0812 03:07:16.018712 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:07:16.019614 5835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:16.008485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:45103 (local address 127.5.96.131:49505)
0812 03:07:16.008908 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:16.008949 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:16.010085 (+ 1136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:16.010529 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:16.010537 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:16.010913 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:16.011367 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:16.011380 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.012143 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.012145 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:16.012996 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:16.013006 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.013391 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.013394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:16.013547 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:16.014082 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:16.014098 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:16.015745 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:16.017338 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:16.017343 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:16.017355 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:16.017598 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:16.017812 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:16.017814 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:16.017816 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:16.017936 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:16.018225 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:16.018230 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:16.018439 (+ 209us) client_negotiation.cc:769] Sending connection context
0812 03:07:16.018582 (+ 143us) client_negotiation.cc:240] Negotiation successful
0812 03:07:16.018816 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":121,"threads_started":1}
I20260812 03:07:16.020649 5830 heartbeater.cc:347] Connected to a master server at 127.5.96.190:45103
I20260812 03:07:16.020870 5830 heartbeater.cc:464] Registering TS with master...
I20260812 03:07:16.021311 5830 heartbeater.cc:511] Master 127.5.96.190:45103 requested a full tablet report, sending...
I20260812 03:07:16.022284 5127 ts_manager.cc:194] Registered new tserver with Master: 27101ddbad354f99bc0cf3dc2464974b (127.5.96.131:32779)
I20260812 03:07:16.022833 5127 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.96.131@KRBTEST.COM'} at 127.5.96.131:49505
I20260812 03:07:16.027684 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:07:16.035795 5183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:16.029514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56710 (local address 127.5.96.190:45103)
0812 03:07:16.029621 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:16.029625 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:16.029827 (+ 202us) server_negotiation.cc:407] Connection header received
0812 03:07:16.029924 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:16.029926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:16.029963 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:16.030013 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:16.030622 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.031104 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.031899 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.032101 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.032168 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:16.033313 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:16.033315 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:16.033317 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:16.033337 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:16.034554 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:16.034865 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:16.034868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:16.034869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:16.034932 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:16.035104 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:16.035106 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:16.035107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:16.035289 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:16.035386 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:16.035529 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 03:07:16.035611 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 03:07:16.038153 5127 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:07:16.041029 5127 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:07:16.054132 5855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:16.049424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:45095 (local address 127.0.0.1:35290)
0812 03:07:16.050258 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:16.050268 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:16.050357 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:16.050777 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:16.050780 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:16.050797 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:16.050883 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:16.050889 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.051822 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.051825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:16.052993 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:16.053003 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.053768 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.053771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:16.053852 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:16.053879 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 03:07:16.053952 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 03:07:16.054030 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"thread_start_us":91,"threads_started":1}
I20260812 03:07:16.054243 5857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:16.049837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35290 (local address 127.5.96.130:45095)
0812 03:07:16.050163 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:16.050175 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:16.050345 (+ 170us) server_negotiation.cc:407] Connection header received
0812 03:07:16.050494 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:16.050499 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:16.050670 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:16.050809 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:16.050957 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.051699 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.053104 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.053656 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.053768 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:16.053911 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:16.053993 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 03:07:16.054079 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":134,"threads_started":1}
I20260812 03:07:16.054872 5854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:16.049079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:35345 (local address 127.0.0.1:57724)
0812 03:07:16.049664 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:16.049676 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:16.049767 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:16.050451 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:16.050457 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:16.050494 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:16.050615 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:16.050621 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.052180 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.052184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:16.053572 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:16.053583 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.054590 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.054593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:16.054658 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:16.054677 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 03:07:16.054722 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 03:07:16.054775 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"mutex_wait_us":109,"spinlock_wait_cycles":7296,"thread_start_us":97,"threads_started":1}
I20260812 03:07:16.054984 5856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:16.049500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57724 (local address 127.5.96.129:35345)
0812 03:07:16.049908 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:16.049928 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:16.049952 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:07:16.050030 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:16.050035 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:16.050319 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:16.050460 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:16.050740 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.052031 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.053703 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.054482 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.054587 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:16.054721 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:16.054766 (+ 45us) server_negotiation.cc:299] Negotiation successful
0812 03:07:16.054841 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":115,"threads_started":1}
I20260812 03:07:16.056469 5850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:16.048749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:32779 (local address 127.0.0.1:54454)
0812 03:07:16.049052 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:16.049101 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:16.049367 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:16.050454 (+ 1087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:16.050457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:16.050497 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:16.050622 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:16.050629 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.052180 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.052184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:16.053572 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:16.053583 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.056111 (+ 2528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.056115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:16.056205 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:16.056228 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:07:16.056275 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 03:07:16.056340 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":119,"spinlock_wait_cycles":4096,"thread_start_us":60,"threads_started":1}
I20260812 03:07:16.057062 5853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:16.048923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54454 (local address 127.5.96.131:32779)
0812 03:07:16.049994 (+ 1071us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:16.050004 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:16.050021 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:07:16.050079 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:16.050083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:16.050279 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:16.050775 (+ 496us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:16.050928 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.052038 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.055169 (+ 3131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.055970 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.056652 (+ 682us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:16.056797 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:16.056845 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 03:07:16.056925 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":986,"thread_start_us":156,"threads_started":1}
I20260812 03:07:16.057272 5520 tablet_service.cc:1467] Processing CreateTablet for tablet 1145ad2c705943f2b09e448dc20a8e09 (DEFAULT_TABLE table=test-table [id=8cd0cdb338c8469688eae737f38fc891]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:07:16.057475 5315 tablet_service.cc:1467] Processing CreateTablet for tablet 1145ad2c705943f2b09e448dc20a8e09 (DEFAULT_TABLE table=test-table [id=8cd0cdb338c8469688eae737f38fc891]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:07:16.058204 5520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1145ad2c705943f2b09e448dc20a8e09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:16.058362 5315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1145ad2c705943f2b09e448dc20a8e09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:16.063930 5859 tablet_bootstrap.cc:492] T 1145ad2c705943f2b09e448dc20a8e09 P c738b378e4b6448b95ecb3f444a037d5: Bootstrap starting.
I20260812 03:07:16.065204 5861 tablet_bootstrap.cc:492] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53: Bootstrap starting.
I20260812 03:07:16.066375 5736 tablet_service.cc:1467] Processing CreateTablet for tablet 1145ad2c705943f2b09e448dc20a8e09 (DEFAULT_TABLE table=test-table [id=8cd0cdb338c8469688eae737f38fc891]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:07:16.066854 5861 tablet_bootstrap.cc:654] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:16.066956 5859 tablet_bootstrap.cc:654] T 1145ad2c705943f2b09e448dc20a8e09 P c738b378e4b6448b95ecb3f444a037d5: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:16.067428 5736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1145ad2c705943f2b09e448dc20a8e09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:16.068007 5861 log.cc:824] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:16.068209 5859 log.cc:824] T 1145ad2c705943f2b09e448dc20a8e09 P c738b378e4b6448b95ecb3f444a037d5: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:16.069828 5859 tablet_bootstrap.cc:492] T 1145ad2c705943f2b09e448dc20a8e09 P c738b378e4b6448b95ecb3f444a037d5: No bootstrap required, opened a new log
I20260812 03:07:16.069895 5861 tablet_bootstrap.cc:492] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53: No bootstrap required, opened a new log
I20260812 03:07:16.070063 5861 ts_tablet_manager.cc:1397] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 03:07:16.070079 5859 ts_tablet_manager.cc:1397] T 1145ad2c705943f2b09e448dc20a8e09 P c738b378e4b6448b95ecb3f444a037d5: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.003s
I20260812 03:07:16.072926 5859 raft_consensus.cc:348] T 1145ad2c705943f2b09e448dc20a8e09 P c738b378e4b6448b95ecb3f444a037d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27101ddbad354f99bc0cf3dc2464974b" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 32779 } } peers { permanent_uuid: "f6bb443eacc545b9b8edd2b100457f53" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 45095 } } peers { permanent_uuid: "c738b378e4b6448b95ecb3f444a037d5" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 35345 } }
I20260812 03:07:16.073125 5859 raft_consensus.cc:374] T 1145ad2c705943f2b09e448dc20a8e09 P c738b378e4b6448b95ecb3f444a037d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:16.073179 5859 raft_consensus.cc:729] T 1145ad2c705943f2b09e448dc20a8e09 P c738b378e4b6448b95ecb3f444a037d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c738b378e4b6448b95ecb3f444a037d5, State: Initialized, Role: FOLLOWER
I20260812 03:07:16.073604 5859 consensus_queue.cc:260] T 1145ad2c705943f2b09e448dc20a8e09 P c738b378e4b6448b95ecb3f444a037d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27101ddbad354f99bc0cf3dc2464974b" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 32779 } } peers { permanent_uuid: "f6bb443eacc545b9b8edd2b100457f53" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 45095 } } peers { permanent_uuid: "c738b378e4b6448b95ecb3f444a037d5" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 35345 } }
I20260812 03:07:16.073788 5861 raft_consensus.cc:348] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27101ddbad354f99bc0cf3dc2464974b" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 32779 } } peers { permanent_uuid: "f6bb443eacc545b9b8edd2b100457f53" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 45095 } } peers { permanent_uuid: "c738b378e4b6448b95ecb3f444a037d5" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 35345 } }
I20260812 03:07:16.074050 5861 raft_consensus.cc:374] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:16.074107 5861 raft_consensus.cc:729] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bb443eacc545b9b8edd2b100457f53, State: Initialized, Role: FOLLOWER
I20260812 03:07:16.074322 5419 heartbeater.cc:503] Master 127.5.96.190:45103 was elected leader, sending a full tablet report...
I20260812 03:07:16.074512 5861 consensus_queue.cc:260] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27101ddbad354f99bc0cf3dc2464974b" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 32779 } } peers { permanent_uuid: "f6bb443eacc545b9b8edd2b100457f53" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 45095 } } peers { permanent_uuid: "c738b378e4b6448b95ecb3f444a037d5" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 35345 } }
I20260812 03:07:16.074723 5859 ts_tablet_manager.cc:1428] T 1145ad2c705943f2b09e448dc20a8e09 P c738b378e4b6448b95ecb3f444a037d5: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260812 03:07:16.075054 5865 tablet_bootstrap.cc:492] T 1145ad2c705943f2b09e448dc20a8e09 P 27101ddbad354f99bc0cf3dc2464974b: Bootstrap starting.
I20260812 03:07:16.075390 5861 ts_tablet_manager.cc:1428] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260812 03:07:16.075752 5628 heartbeater.cc:503] Master 127.5.96.190:45103 was elected leader, sending a full tablet report...
I20260812 03:07:16.077050 5865 tablet_bootstrap.cc:654] T 1145ad2c705943f2b09e448dc20a8e09 P 27101ddbad354f99bc0cf3dc2464974b: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:16.078233 5865 log.cc:824] T 1145ad2c705943f2b09e448dc20a8e09 P 27101ddbad354f99bc0cf3dc2464974b: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:16.079905 5865 tablet_bootstrap.cc:492] T 1145ad2c705943f2b09e448dc20a8e09 P 27101ddbad354f99bc0cf3dc2464974b: No bootstrap required, opened a new log
I20260812 03:07:16.080063 5865 ts_tablet_manager.cc:1397] T 1145ad2c705943f2b09e448dc20a8e09 P 27101ddbad354f99bc0cf3dc2464974b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 03:07:16.082610 5865 raft_consensus.cc:348] T 1145ad2c705943f2b09e448dc20a8e09 P 27101ddbad354f99bc0cf3dc2464974b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27101ddbad354f99bc0cf3dc2464974b" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 32779 } } peers { permanent_uuid: "f6bb443eacc545b9b8edd2b100457f53" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 45095 } } peers { permanent_uuid: "c738b378e4b6448b95ecb3f444a037d5" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 35345 } }
I20260812 03:07:16.082803 5865 raft_consensus.cc:374] T 1145ad2c705943f2b09e448dc20a8e09 P 27101ddbad354f99bc0cf3dc2464974b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:16.082863 5865 raft_consensus.cc:729] T 1145ad2c705943f2b09e448dc20a8e09 P 27101ddbad354f99bc0cf3dc2464974b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27101ddbad354f99bc0cf3dc2464974b, State: Initialized, Role: FOLLOWER
I20260812 03:07:16.083233 5865 consensus_queue.cc:260] T 1145ad2c705943f2b09e448dc20a8e09 P 27101ddbad354f99bc0cf3dc2464974b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27101ddbad354f99bc0cf3dc2464974b" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 32779 } } peers { permanent_uuid: "f6bb443eacc545b9b8edd2b100457f53" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 45095 } } peers { permanent_uuid: "c738b378e4b6448b95ecb3f444a037d5" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 35345 } }
I20260812 03:07:16.083786 5830 heartbeater.cc:503] Master 127.5.96.190:45103 was elected leader, sending a full tablet report...
I20260812 03:07:16.084064 5865 ts_tablet_manager.cc:1428] T 1145ad2c705943f2b09e448dc20a8e09 P 27101ddbad354f99bc0cf3dc2464974b: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
W20260812 03:07:16.106467 5420 tablet.cc:2369] T 1145ad2c705943f2b09e448dc20a8e09 P c738b378e4b6448b95ecb3f444a037d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:07:16.192014 5868 raft_consensus.cc:482] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:07:16.192204 5868 raft_consensus.cc:504] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27101ddbad354f99bc0cf3dc2464974b" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 32779 } } peers { permanent_uuid: "f6bb443eacc545b9b8edd2b100457f53" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 45095 } } peers { permanent_uuid: "c738b378e4b6448b95ecb3f444a037d5" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 35345 } }
I20260812 03:07:16.193328 5868 leader_election.cc:288] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27101ddbad354f99bc0cf3dc2464974b (127.5.96.131:32779), c738b378e4b6448b95ecb3f444a037d5 (127.5.96.129:35345)
I20260812 03:07:16.197273 5630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:16.193552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:35345 (local address 127.5.96.130:60577)
0812 03:07:16.193659 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:16.193673 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:16.193759 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:16.193998 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:16.194002 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:16.194023 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:16.194129 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:16.194134 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.194915 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.194919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:16.196152 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:16.196163 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.196916 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.196919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:16.197002 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:16.197024 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:07:16.197079 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 03:07:16.197141 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 03:07:16.197314 5856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:16.193631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:60577 (local address 127.5.96.129:35345)
0812 03:07:16.193751 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:16.193756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:16.193771 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:07:16.193851 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:16.193853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:16.193892 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:16.193962 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:16.194240 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.194780 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.196288 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.196808 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.196925 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:16.196979 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:16.197119 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 03:07:16.197194 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 03:07:16.197633 5853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:16.193631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:42595 (local address 127.5.96.131:32779)
0812 03:07:16.193751 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:16.193756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:16.194097 (+ 341us) server_negotiation.cc:407] Connection header received
0812 03:07:16.194190 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:16.194193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:16.194237 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:16.194299 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:16.194564 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.195167 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.196509 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.197066 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.197177 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:16.197220 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:16.197446 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 03:07:16.197522 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 03:07:16.197700 5874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:16.193552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:32779 (local address 127.5.96.130:42595)
0812 03:07:16.194008 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:16.194019 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:16.194109 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:16.194320 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:16.194322 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:16.194337 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:16.194461 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:16.194466 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.195283 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.195286 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:16.196403 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:16.196411 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.197200 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.197203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:16.197296 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:16.197319 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:07:16.197503 (+ 184us) client_negotiation.cc:240] Negotiation successful
0812 03:07:16.197568 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":105,"thread_start_us":151,"threads_started":1}
I20260812 03:07:16.197906 5347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1145ad2c705943f2b09e448dc20a8e09" candidate_uuid: "f6bb443eacc545b9b8edd2b100457f53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c738b378e4b6448b95ecb3f444a037d5" is_pre_election: true
I20260812 03:07:16.198201 5347 raft_consensus.cc:2454] T 1145ad2c705943f2b09e448dc20a8e09 P c738b378e4b6448b95ecb3f444a037d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6bb443eacc545b9b8edd2b100457f53 in term 0.
I20260812 03:07:16.198254 5770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1145ad2c705943f2b09e448dc20a8e09" candidate_uuid: "f6bb443eacc545b9b8edd2b100457f53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27101ddbad354f99bc0cf3dc2464974b" is_pre_election: true
I20260812 03:07:16.198493 5770 raft_consensus.cc:2454] T 1145ad2c705943f2b09e448dc20a8e09 P 27101ddbad354f99bc0cf3dc2464974b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6bb443eacc545b9b8edd2b100457f53 in term 0.
I20260812 03:07:16.198693 5452 leader_election.cc:302] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c738b378e4b6448b95ecb3f444a037d5, f6bb443eacc545b9b8edd2b100457f53; no voters:
I20260812 03:07:16.198961 5868 raft_consensus.cc:2781] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:07:16.199040 5868 raft_consensus.cc:482] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:07:16.199070 5868 raft_consensus.cc:3037] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:16.200080 5868 raft_consensus.cc:504] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27101ddbad354f99bc0cf3dc2464974b" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 32779 } } peers { permanent_uuid: "f6bb443eacc545b9b8edd2b100457f53" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 45095 } } peers { permanent_uuid: "c738b378e4b6448b95ecb3f444a037d5" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 35345 } }
I20260812 03:07:16.200392 5868 leader_election.cc:288] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53 [CANDIDATE]: Term 1 election: Requested vote from peers 27101ddbad354f99bc0cf3dc2464974b (127.5.96.131:32779), c738b378e4b6448b95ecb3f444a037d5 (127.5.96.129:35345)
I20260812 03:07:16.200735 5770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1145ad2c705943f2b09e448dc20a8e09" candidate_uuid: "f6bb443eacc545b9b8edd2b100457f53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27101ddbad354f99bc0cf3dc2464974b"
I20260812 03:07:16.200735 5347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1145ad2c705943f2b09e448dc20a8e09" candidate_uuid: "f6bb443eacc545b9b8edd2b100457f53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c738b378e4b6448b95ecb3f444a037d5"
I20260812 03:07:16.200856 5770 raft_consensus.cc:3037] T 1145ad2c705943f2b09e448dc20a8e09 P 27101ddbad354f99bc0cf3dc2464974b [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:16.200856 5347 raft_consensus.cc:3037] T 1145ad2c705943f2b09e448dc20a8e09 P c738b378e4b6448b95ecb3f444a037d5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:16.202039 5347 raft_consensus.cc:2454] T 1145ad2c705943f2b09e448dc20a8e09 P c738b378e4b6448b95ecb3f444a037d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6bb443eacc545b9b8edd2b100457f53 in term 1.
I20260812 03:07:16.202039 5770 raft_consensus.cc:2454] T 1145ad2c705943f2b09e448dc20a8e09 P 27101ddbad354f99bc0cf3dc2464974b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6bb443eacc545b9b8edd2b100457f53 in term 1.
I20260812 03:07:16.202361 5455 leader_election.cc:302] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 27101ddbad354f99bc0cf3dc2464974b, c738b378e4b6448b95ecb3f444a037d5, f6bb443eacc545b9b8edd2b100457f53; no voters:
I20260812 03:07:16.202528 5868 raft_consensus.cc:2781] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:16.202744 5868 raft_consensus.cc:686] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53 [term 1 LEADER]: Becoming Leader. State: Replica: f6bb443eacc545b9b8edd2b100457f53, State: Running, Role: LEADER
I20260812 03:07:16.203030 5868 consensus_queue.cc:237] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27101ddbad354f99bc0cf3dc2464974b" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 32779 } } peers { permanent_uuid: "f6bb443eacc545b9b8edd2b100457f53" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 45095 } } peers { permanent_uuid: "c738b378e4b6448b95ecb3f444a037d5" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 35345 } }
I20260812 03:07:16.205946 5127 catalog_manager.cc:5495] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6bb443eacc545b9b8edd2b100457f53 (127.5.96.130). New cstate: current_term: 1 leader_uuid: "f6bb443eacc545b9b8edd2b100457f53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27101ddbad354f99bc0cf3dc2464974b" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 32779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6bb443eacc545b9b8edd2b100457f53" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c738b378e4b6448b95ecb3f444a037d5" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 35345 } health_report { overall_health: UNKNOWN } } }
I20260812 03:07:16.239015 5857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:16.236068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35300 (local address 127.5.96.130:45095)
0812 03:07:16.236186 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:16.236189 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:16.236202 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:07:16.236314 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:16.236316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:16.236368 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:16.236456 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:07:16.236639 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.237113 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.237877 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.238080 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.238142 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:16.238355 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:07:16.238734 (+ 379us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:07:16.238817 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:16.238849 (+ 32us) server_negotiation.cc:299] Negotiation successful
0812 03:07:16.238897 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 03:07:16.244513 5770 raft_consensus.cc:1261] T 1145ad2c705943f2b09e448dc20a8e09 P 27101ddbad354f99bc0cf3dc2464974b [term 1 FOLLOWER]: Refusing update from remote peer f6bb443eacc545b9b8edd2b100457f53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:07:16.244513 5347 raft_consensus.cc:1261] T 1145ad2c705943f2b09e448dc20a8e09 P c738b378e4b6448b95ecb3f444a037d5 [term 1 FOLLOWER]: Refusing update from remote peer f6bb443eacc545b9b8edd2b100457f53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:07:16.245191 5876 consensus_queue.cc:1035] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27101ddbad354f99bc0cf3dc2464974b" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 32779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:07:16.245402 5868 consensus_queue.cc:1035] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c738b378e4b6448b95ecb3f444a037d5" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 35345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:07:16.252664 5881 mvcc.cc:205] Tried to move back new op lower bound from 7317520532450680832 to 7317520532289929216. Current Snapshot: MvccSnapshot[applied={T|T < 7317520532450680832}]
I20260812 03:07:16.254827 5880 mvcc.cc:205] Tried to move back new op lower bound from 7317520532450680832 to 7317520532289929216. Current Snapshot: MvccSnapshot[applied={T|T < 7317520532450680832 or (T in {7317520532450680832})}]
I20260812 03:07:16.259593 5124 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:07:16.259758 5124 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:07:16.261994 5124 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79c7343281784981af3c2a1ea03102ca: Sending DeleteTablet for 3 replicas of tablet 1145ad2c705943f2b09e448dc20a8e09
I20260812 03:07:16.262764 5315 tablet_service.cc:1514] Processing DeleteTablet for tablet 1145ad2c705943f2b09e448dc20a8e09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:07:16 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:57724
I20260812 03:07:16.262866 5736 tablet_service.cc:1514] Processing DeleteTablet for tablet 1145ad2c705943f2b09e448dc20a8e09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:07:16 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:54454
I20260812 03:07:16.263239 5892 tablet_replica.cc:330] T 1145ad2c705943f2b09e448dc20a8e09 P c738b378e4b6448b95ecb3f444a037d5: stopping tablet replica
I20260812 03:07:16.263303 5893 tablet_replica.cc:330] T 1145ad2c705943f2b09e448dc20a8e09 P 27101ddbad354f99bc0cf3dc2464974b: stopping tablet replica
I20260812 03:07:16.263411 5520 tablet_service.cc:1514] Processing DeleteTablet for tablet 1145ad2c705943f2b09e448dc20a8e09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:07:16 UTC) from {username='kudu', principal='kudu/127.5.96.190@KRBTEST.COM'} at 127.0.0.1:35290
I20260812 03:07:16.263592 5893 raft_consensus.cc:2229] T 1145ad2c705943f2b09e448dc20a8e09 P 27101ddbad354f99bc0cf3dc2464974b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:07:16.263733 5892 raft_consensus.cc:2229] T 1145ad2c705943f2b09e448dc20a8e09 P c738b378e4b6448b95ecb3f444a037d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:07:16.263898 5893 raft_consensus.cc:2258] T 1145ad2c705943f2b09e448dc20a8e09 P 27101ddbad354f99bc0cf3dc2464974b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:07:16.264089 5892 raft_consensus.cc:2258] T 1145ad2c705943f2b09e448dc20a8e09 P c738b378e4b6448b95ecb3f444a037d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:07:16.264102 5894 tablet_replica.cc:330] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53: stopping tablet replica
I20260812 03:07:16.264216 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 5220
I20260812 03:07:16.264338 5894 raft_consensus.cc:2229] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:07:16.264628 5894 raft_consensus.cc:2258] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:07:16.265774 5894 ts_tablet_manager.cc:1905] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:07:16.267853 5894 ts_tablet_manager.cc:1918] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:07:16.267937 5894 log.cc:1196] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-1/wal/wals/1145ad2c705943f2b09e448dc20a8e09
I20260812 03:07:16.268211 5894 ts_tablet_manager.cc:1939] T 1145ad2c705943f2b09e448dc20a8e09 P f6bb443eacc545b9b8edd2b100457f53: Deleting consensus metadata
I20260812 03:07:16.268337 5893 ts_tablet_manager.cc:1905] T 1145ad2c705943f2b09e448dc20a8e09 P 27101ddbad354f99bc0cf3dc2464974b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:07:16.269140 5108 catalog_manager.cc:4841] TS f6bb443eacc545b9b8edd2b100457f53 (127.5.96.130:45095): tablet 1145ad2c705943f2b09e448dc20a8e09 (table test-table [id=8cd0cdb338c8469688eae737f38fc891]) successfully deleted
I20260812 03:07:16.271310 5893 ts_tablet_manager.cc:1918] T 1145ad2c705943f2b09e448dc20a8e09 P 27101ddbad354f99bc0cf3dc2464974b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:07:16.271433 5893 log.cc:1196] T 1145ad2c705943f2b09e448dc20a8e09 P 27101ddbad354f99bc0cf3dc2464974b: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786503790637001-5506-0/minicluster-data/ts-2/wal/wals/1145ad2c705943f2b09e448dc20a8e09
I20260812 03:07:16.271862 5893 ts_tablet_manager.cc:1939] T 1145ad2c705943f2b09e448dc20a8e09 P 27101ddbad354f99bc0cf3dc2464974b: Deleting consensus metadata
I20260812 03:07:16.272913 5108 catalog_manager.cc:4841] TS 27101ddbad354f99bc0cf3dc2464974b (127.5.96.131:32779): tablet 1145ad2c705943f2b09e448dc20a8e09 (table test-table [id=8cd0cdb338c8469688eae737f38fc891]) successfully deleted
W20260812 03:07:16.287021 5103 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.96.129:35345 (error 108)
I20260812 03:07:16.287137 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 5427
I20260812 03:07:16.290248 5850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:16.289127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:35345 (local address 127.0.0.1:57740)
0812 03:07:16.290109 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:16.290172 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.129:35345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":908}
W20260812 03:07:16.290756 5103 catalog_manager.cc:4584] TS c738b378e4b6448b95ecb3f444a037d5 (127.5.96.129:35345): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1145ad2c705943f2b09e448dc20a8e09: Network error: Client connection negotiation failed: client connection to 127.5.96.129:35345: connect: Connection refused (error 111)
I20260812 03:07:16.295596 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 5635
I20260812 03:07:16.306133 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 5075
2026-08-12T03:07:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2453 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T03:07:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:16Z Disabled control of system clock
I20260812 03:07:16.341018 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:33911
--webserver_interface=127.5.96.190
--webserver_port=0
--builtin_ntp_servers=127.5.96.148:33491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:33911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:07:16.448990 5902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:16.449230 5902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:16.449265 5902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:16.453156 5902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:16.453202 5902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:16.453218 5902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:16.453248 5902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:16.453271 5902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:16.457902 5902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:33491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:33911
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:33911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:16.458851 5902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:16.460351 5902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:16.468210 5911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:16.468202 5910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:16.468329 5914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:16.468557 5902 server_base.cc:1034] running on GCE node
I20260812 03:07:16.469095 5902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:16.470108 5902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:16.471256 5902 hybrid_clock.cc:648] HybridClock initialized: now 1786504036471236 us; error 35 us; skew 500 ppm
I20260812 03:07:16.472972 5902 webserver.cc:460] Webserver started at http://127.5.96.190:40071/ using document root <none> and password file <none>
I20260812 03:07:16.473479 5902 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:16.473546 5902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:16.473729 5902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:16.475601 5902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "a88afbe3635d4c84809e58dec3f4d759"
format_stamp: "Formatted at 2026-08-12 03:07:16 on dist-test-slave-6bbx"
server_key: "ec638afd82d1011418a48edfd951a4e7"
server_key_iv: "0150926f6e63475ba900c98601b64c9f"
server_key_version: "encryptionkey@0"
I20260812 03:07:16.476166 5902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "a88afbe3635d4c84809e58dec3f4d759"
format_stamp: "Formatted at 2026-08-12 03:07:16 on dist-test-slave-6bbx"
server_key: "ec638afd82d1011418a48edfd951a4e7"
server_key_iv: "0150926f6e63475ba900c98601b64c9f"
server_key_version: "encryptionkey@0"
I20260812 03:07:16.479449 5902 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:07:16.481741 5922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:16.482859 5902 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 03:07:16.482966 5902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "a88afbe3635d4c84809e58dec3f4d759"
format_stamp: "Formatted at 2026-08-12 03:07:16 on dist-test-slave-6bbx"
server_key: "ec638afd82d1011418a48edfd951a4e7"
server_key_iv: "0150926f6e63475ba900c98601b64c9f"
server_key_version: "encryptionkey@0"
I20260812 03:07:16.483050 5902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:16.494230 5902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:16.494787 5902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:16.495097 5902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:16.501662 5902 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:33911
I20260812 03:07:16.501674 5999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:33911 every 8 connection(s)
I20260812 03:07:16.502519 5902 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:16.505447 6003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:16.506073 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 5902
I20260812 03:07:16.506191 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:07:16.506496 5506 external_mini_cluster.cc:1428] Setting key c649a0d7a8fb2b3e328ea4f5f37b8ecd
I20260812 03:07:16.511293 6003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a88afbe3635d4c84809e58dec3f4d759: Bootstrap starting.
I20260812 03:07:16.513854 6003 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a88afbe3635d4c84809e58dec3f4d759: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:16.514704 6003 log.cc:824] T 00000000000000000000000000000000 P a88afbe3635d4c84809e58dec3f4d759: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:16.515594 6006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:16.507860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49324 (local address 127.5.96.190:33911)
0812 03:07:16.508427 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:16.508442 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:16.508475 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:07:16.509091 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:16.509106 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:16.509484 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:16.509746 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:16.510053 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.510957 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.512180 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.512865 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.513015 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:16.513729 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:16.513736 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:16.513751 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:16.514220 (+ 469us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:16.514288 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:16.514315 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:16.514412 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:16.514530 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 03:07:16.514795 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":102,"threads_started":1}
I20260812 03:07:16.516620 6003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a88afbe3635d4c84809e58dec3f4d759: No bootstrap required, opened a new log
I20260812 03:07:16.520188 6003 raft_consensus.cc:348] T 00000000000000000000000000000000 P a88afbe3635d4c84809e58dec3f4d759 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a88afbe3635d4c84809e58dec3f4d759" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33911 } }
I20260812 03:07:16.520444 6003 raft_consensus.cc:374] T 00000000000000000000000000000000 P a88afbe3635d4c84809e58dec3f4d759 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:16.520503 6003 raft_consensus.cc:729] T 00000000000000000000000000000000 P a88afbe3635d4c84809e58dec3f4d759 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a88afbe3635d4c84809e58dec3f4d759, State: Initialized, Role: FOLLOWER
I20260812 03:07:16.520897 6003 consensus_queue.cc:260] T 00000000000000000000000000000000 P a88afbe3635d4c84809e58dec3f4d759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a88afbe3635d4c84809e58dec3f4d759" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33911 } }
I20260812 03:07:16.521018 6003 raft_consensus.cc:388] T 00000000000000000000000000000000 P a88afbe3635d4c84809e58dec3f4d759 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:07:16.521062 6003 raft_consensus.cc:482] T 00000000000000000000000000000000 P a88afbe3635d4c84809e58dec3f4d759 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:07:16.521129 6003 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a88afbe3635d4c84809e58dec3f4d759 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:16.521860 6003 raft_consensus.cc:504] T 00000000000000000000000000000000 P a88afbe3635d4c84809e58dec3f4d759 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a88afbe3635d4c84809e58dec3f4d759" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33911 } }
I20260812 03:07:16.522150 6003 leader_election.cc:302] T 00000000000000000000000000000000 P a88afbe3635d4c84809e58dec3f4d759 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a88afbe3635d4c84809e58dec3f4d759; no voters:
I20260812 03:07:16.522385 6003 leader_election.cc:288] T 00000000000000000000000000000000 P a88afbe3635d4c84809e58dec3f4d759 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:07:16.522482 6011 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a88afbe3635d4c84809e58dec3f4d759 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:16.522683 6011 raft_consensus.cc:686] T 00000000000000000000000000000000 P a88afbe3635d4c84809e58dec3f4d759 [term 1 LEADER]: Becoming Leader. State: Replica: a88afbe3635d4c84809e58dec3f4d759, State: Running, Role: LEADER
I20260812 03:07:16.522967 6011 consensus_queue.cc:237] T 00000000000000000000000000000000 P a88afbe3635d4c84809e58dec3f4d759 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a88afbe3635d4c84809e58dec3f4d759" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33911 } }
I20260812 03:07:16.523432 6003 sys_catalog.cc:564] T 00000000000000000000000000000000 P a88afbe3635d4c84809e58dec3f4d759 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:16.524446 6013 sys_catalog.cc:455] T 00000000000000000000000000000000 P a88afbe3635d4c84809e58dec3f4d759 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a88afbe3635d4c84809e58dec3f4d759. Latest consensus state: current_term: 1 leader_uuid: "a88afbe3635d4c84809e58dec3f4d759" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a88afbe3635d4c84809e58dec3f4d759" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33911 } } }
I20260812 03:07:16.524573 6013 sys_catalog.cc:458] T 00000000000000000000000000000000 P a88afbe3635d4c84809e58dec3f4d759 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:16.524855 6012 sys_catalog.cc:455] T 00000000000000000000000000000000 P a88afbe3635d4c84809e58dec3f4d759 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a88afbe3635d4c84809e58dec3f4d759" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a88afbe3635d4c84809e58dec3f4d759" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 33911 } } }
I20260812 03:07:16.524924 6027 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:16.524952 6012 sys_catalog.cc:458] T 00000000000000000000000000000000 P a88afbe3635d4c84809e58dec3f4d759 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:16.528008 6027 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:16.533212 6027 catalog_manager.cc:1347] Generated new cluster ID: 37070b0874e94dd995d14c02305e6577
I20260812 03:07:16.533272 6027 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:16.559691 6027 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:16.560493 6027 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:16.563746 6027 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a88afbe3635d4c84809e58dec3f4d759: Generated new TSK 0
I20260812 03:07:16.564294 6027 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:07:16.568557 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.129:0
--local_ip_for_outbound_sockets=127.5.96.129
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:33911
--builtin_ntp_servers=127.5.96.148:33491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:07:16.678207 6040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:16.678421 6040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:16.678457 6040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:16.682292 6040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:16.682335 6040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:16.682397 6040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.129
I20260812 03:07:16.687002 6040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:33491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.96.129
--webserver_port=0
--tserver_master_addrs=127.5.96.190:33911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.129
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:16.687942 6040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:16.689190 6040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:16.697961 6047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:16.697990 6052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:16.697974 6048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:16.698078 6040 server_base.cc:1034] running on GCE node
I20260812 03:07:16.698591 6040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:16.699143 6040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:16.700291 6040 hybrid_clock.cc:648] HybridClock initialized: now 1786504036700278 us; error 26 us; skew 500 ppm
I20260812 03:07:16.701898 6040 webserver.cc:460] Webserver started at http://127.5.96.129:38247/ using document root <none> and password file <none>
I20260812 03:07:16.702404 6040 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:16.702461 6040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:16.702615 6040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:16.704475 6040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-0/data/instance:
uuid: "a2b9cd6122a9414d8764a6ba2ddd50f9"
format_stamp: "Formatted at 2026-08-12 03:07:16 on dist-test-slave-6bbx"
server_key: "507ccf3ea3f1c798397c63226400d305"
server_key_iv: "25cdef487ca2d70ca9f32334051b93b9"
server_key_version: "encryptionkey@0"
I20260812 03:07:16.704948 6040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance:
uuid: "a2b9cd6122a9414d8764a6ba2ddd50f9"
format_stamp: "Formatted at 2026-08-12 03:07:16 on dist-test-slave-6bbx"
server_key: "507ccf3ea3f1c798397c63226400d305"
server_key_iv: "25cdef487ca2d70ca9f32334051b93b9"
server_key_version: "encryptionkey@0"
I20260812 03:07:16.708261 6040 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:07:16.710470 6062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:16.711577 6040 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:07:16.711696 6040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-0/wal
uuid: "a2b9cd6122a9414d8764a6ba2ddd50f9"
format_stamp: "Formatted at 2026-08-12 03:07:16 on dist-test-slave-6bbx"
server_key: "507ccf3ea3f1c798397c63226400d305"
server_key_iv: "25cdef487ca2d70ca9f32334051b93b9"
server_key_version: "encryptionkey@0"
I20260812 03:07:16.711790 6040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:16.743086 6040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:16.743628 6040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:16.743796 6040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:16.744292 6040 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:07:16.745117 6040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:07:16.745170 6040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:16.745206 6040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:07:16.745226 6040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:16.753360 6040 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.129:41833
I20260812 03:07:16.753391 6228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.129:41833 every 8 connection(s)
I20260812 03:07:16.754128 6040 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-0/data/info.pb
I20260812 03:07:16.755390 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 6040
I20260812 03:07:16.755507 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-0/wal/instance
I20260812 03:07:16.755860 5506 external_mini_cluster.cc:1428] Setting key 7a56e51489dbedb2135649084e2af92f
I20260812 03:07:16.757958 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.130:0
--local_ip_for_outbound_sockets=127.5.96.130
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:33911
--builtin_ntp_servers=127.5.96.148:33491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 03:07:16.763695 6006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:16.755929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.129:49835 (local address 127.5.96.190:33911)
0812 03:07:16.756056 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:16.756061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:16.757210 (+ 1149us) server_negotiation.cc:407] Connection header received
0812 03:07:16.758780 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:16.758783 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:16.758828 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:16.758891 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:16.759977 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.760734 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.761806 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.762133 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.762234 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:16.763165 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:16.763169 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:16.763172 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:16.763186 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:16.763210 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:16.763221 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:16.763346 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:16.763449 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 03:07:16.763516 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:07:16.764408 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:16.756385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33911 (local address 127.5.96.129:49835)
0812 03:07:16.756968 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:16.757004 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:16.758564 (+ 1560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:16.759074 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:16.759084 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:16.759180 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:16.759829 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:16.759850 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.760845 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.760847 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:16.761681 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:16.761689 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.762267 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.762270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:16.762394 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:16.762925 (+ 531us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:07:16.762926 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:16.762963 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:07:16.763022 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:16.763338 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:16.763342 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:16.763346 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:07:16.763426 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 03:07:16.763622 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":118,"threads_started":1}
I20260812 03:07:16.765430 6230 heartbeater.cc:347] Connected to a master server at 127.5.96.190:33911
I20260812 03:07:16.765633 6230 heartbeater.cc:464] Registering TS with master...
I20260812 03:07:16.766093 6230 heartbeater.cc:511] Master 127.5.96.190:33911 requested a full tablet report, sending...
I20260812 03:07:16.767412 5950 ts_manager.cc:194] Registered new tserver with Master: a2b9cd6122a9414d8764a6ba2ddd50f9 (127.5.96.129:41833)
I20260812 03:07:16.768525 5950 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.96.129:49835
W20260812 03:07:16.866838 6235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:16.867059 6235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:16.867111 6235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:16.870962 6235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:16.871007 6235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:16.871078 6235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.130
I20260812 03:07:16.875592 6235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:33491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.96.130
--webserver_port=0
--tserver_master_addrs=127.5.96.190:33911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.130
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:16.876487 6235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:16.877699 6235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:16.886474 6243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:16.886479 6245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:16.886482 6248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:16.886718 6235 server_base.cc:1034] running on GCE node
I20260812 03:07:16.887169 6235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:16.887749 6235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:16.888904 6235 hybrid_clock.cc:648] HybridClock initialized: now 1786504036888892 us; error 27 us; skew 500 ppm
I20260812 03:07:16.890550 6235 webserver.cc:460] Webserver started at http://127.5.96.130:36607/ using document root <none> and password file <none>
I20260812 03:07:16.891059 6235 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:16.891124 6235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:16.891330 6235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:16.893268 6235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-1/data/instance:
uuid: "3f22c87b1a8d4bb6b09646962a405c57"
format_stamp: "Formatted at 2026-08-12 03:07:16 on dist-test-slave-6bbx"
server_key: "17760f0eca94f6cc4d31d9fd6544a27f"
server_key_iv: "877ecdad24d52d10185551921aabe743"
server_key_version: "encryptionkey@0"
I20260812 03:07:16.893781 6235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance:
uuid: "3f22c87b1a8d4bb6b09646962a405c57"
format_stamp: "Formatted at 2026-08-12 03:07:16 on dist-test-slave-6bbx"
server_key: "17760f0eca94f6cc4d31d9fd6544a27f"
server_key_iv: "877ecdad24d52d10185551921aabe743"
server_key_version: "encryptionkey@0"
I20260812 03:07:16.897208 6235 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260812 03:07:16.899338 6262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:16.900466 6235 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 03:07:16.900569 6235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-1/wal
uuid: "3f22c87b1a8d4bb6b09646962a405c57"
format_stamp: "Formatted at 2026-08-12 03:07:16 on dist-test-slave-6bbx"
server_key: "17760f0eca94f6cc4d31d9fd6544a27f"
server_key_iv: "877ecdad24d52d10185551921aabe743"
server_key_version: "encryptionkey@0"
I20260812 03:07:16.900663 6235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:16.911612 6235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:16.912165 6235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:16.912312 6235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:16.912825 6235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:07:16.913667 6235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:07:16.913714 6235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:16.913762 6235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:07:16.913790 6235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:16.922202 6235 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.130:35867
I20260812 03:07:16.922230 6424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.130:35867 every 8 connection(s)
I20260812 03:07:16.922999 6235 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-1/data/info.pb
I20260812 03:07:16.924978 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 6235
I20260812 03:07:16.925103 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-1/wal/instance
I20260812 03:07:16.925407 5506 external_mini_cluster.cc:1428] Setting key 3d5c2524e0bedce6671bf3d74f6e8855
I20260812 03:07:16.927440 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.96.131:0
--local_ip_for_outbound_sockets=127.5.96.131
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:33911
--builtin_ntp_servers=127.5.96.148:33491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 03:07:16.932106 6006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:16.924701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:42197 (local address 127.5.96.190:33911)
0812 03:07:16.924854 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:16.924859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:16.925994 (+ 1135us) server_negotiation.cc:407] Connection header received
0812 03:07:16.927492 (+ 1498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:16.927495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:16.927547 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:16.927626 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:16.928661 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.929168 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.930262 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.930478 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.930566 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:16.931642 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:16.931646 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:16.931648 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:16.931659 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:16.931706 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:16.931717 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:16.931779 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:16.931937 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 03:07:16.931992 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 03:07:16.933135 6428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:16.925100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33911 (local address 127.5.96.130:42197)
0812 03:07:16.925856 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:16.925896 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:16.927260 (+ 1364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:16.927812 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:16.927821 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:16.927916 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:16.928521 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:16.928539 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.929301 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.929304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:16.930140 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:16.930149 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:16.930615 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:16.930620 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:16.930802 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:16.931357 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:07:16.931360 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:16.931409 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:07:16.931481 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:16.931794 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:16.931803 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:16.931809 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:07:16.931924 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 03:07:16.932176 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":117,"threads_started":1}
I20260812 03:07:16.934190 6425 heartbeater.cc:347] Connected to a master server at 127.5.96.190:33911
I20260812 03:07:16.934430 6425 heartbeater.cc:464] Registering TS with master...
I20260812 03:07:16.934999 6425 heartbeater.cc:511] Master 127.5.96.190:33911 requested a full tablet report, sending...
I20260812 03:07:16.936056 5950 ts_manager.cc:194] Registered new tserver with Master: 3f22c87b1a8d4bb6b09646962a405c57 (127.5.96.130:35867)
I20260812 03:07:16.936618 5950 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.96.130:42197
W20260812 03:07:17.037297 6430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:17.037528 6430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:17.037578 6430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:17.041458 6430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:17.041502 6430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:17.041574 6430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.96.131
I20260812 03:07:17.046099 6430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:33491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.96.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.96.131
--webserver_port=0
--tserver_master_addrs=127.5.96.190:33911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.96.131
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:17.047004 6430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:17.048275 6430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:17.057053 6439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:17.057044 6442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:17.057044 6437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:17.057343 6430 server_base.cc:1034] running on GCE node
I20260812 03:07:17.057688 6430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:17.058205 6430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:17.059357 6430 hybrid_clock.cc:648] HybridClock initialized: now 1786504037059343 us; error 23 us; skew 500 ppm
I20260812 03:07:17.061000 6430 webserver.cc:460] Webserver started at http://127.5.96.131:41923/ using document root <none> and password file <none>
I20260812 03:07:17.061492 6430 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:17.061548 6430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:17.061707 6430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:17.063561 6430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-2/data/instance:
uuid: "2cda0537314a4d138637894bfc9fb3c3"
format_stamp: "Formatted at 2026-08-12 03:07:17 on dist-test-slave-6bbx"
server_key: "f124caffb37896c22d7dd82beea91d0c"
server_key_iv: "05272d299589e6dad4f82382ea2a4d9a"
server_key_version: "encryptionkey@0"
I20260812 03:07:17.064116 6430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance:
uuid: "2cda0537314a4d138637894bfc9fb3c3"
format_stamp: "Formatted at 2026-08-12 03:07:17 on dist-test-slave-6bbx"
server_key: "f124caffb37896c22d7dd82beea91d0c"
server_key_iv: "05272d299589e6dad4f82382ea2a4d9a"
server_key_version: "encryptionkey@0"
I20260812 03:07:17.067478 6430 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:07:17.069665 6451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:17.070715 6430 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:07:17.070822 6430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-2/wal
uuid: "2cda0537314a4d138637894bfc9fb3c3"
format_stamp: "Formatted at 2026-08-12 03:07:17 on dist-test-slave-6bbx"
server_key: "f124caffb37896c22d7dd82beea91d0c"
server_key_iv: "05272d299589e6dad4f82382ea2a4d9a"
server_key_version: "encryptionkey@0"
I20260812 03:07:17.070914 6430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:17.086359 6430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:17.086889 6430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:17.087034 6430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:17.087589 6430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:07:17.088464 6430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:07:17.088510 6430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:17.088560 6430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:07:17.088589 6430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:17.096546 6430 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.131:44101
I20260812 03:07:17.096576 6619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.131:44101 every 8 connection(s)
I20260812 03:07:17.097329 6430 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-2/data/info.pb
I20260812 03:07:17.105567 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 6430
I20260812 03:07:17.105674 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-2/wal/instance
I20260812 03:07:17.105958 5506 external_mini_cluster.cc:1428] Setting key db0ee0d59952bce80757f201c4833726
I20260812 03:07:17.105962 6006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:17.098873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.131:53599 (local address 127.5.96.190:33911)
0812 03:07:17.098993 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:17.098996 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:17.099980 (+ 984us) server_negotiation.cc:407] Connection header received
0812 03:07:17.101299 (+ 1319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:17.101303 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:17.101360 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:17.101432 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:17.102278 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.102789 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.103922 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.104174 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.104275 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:17.105160 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:17.105163 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:17.105166 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:17.105179 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:17.105199 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:17.105206 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:17.105277 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:17.105563 (+ 286us) server_negotiation.cc:299] Negotiation successful
0812 03:07:17.105745 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:07:17.106653 6624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:17.099206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:33911 (local address 127.5.96.131:53599)
0812 03:07:17.099851 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:17.099889 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:17.101101 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:17.101599 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:17.101606 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:17.101686 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:17.102139 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:17.102151 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.102906 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.102909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:17.103811 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:17.103821 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.104312 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.104317 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:17.104464 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:17.104930 (+ 466us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:07:17.104931 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:17.104967 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:07:17.105023 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:17.105324 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:17.105335 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:17.105342 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 03:07:17.105529 (+ 187us) client_negotiation.cc:240] Negotiation successful
0812 03:07:17.105809 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":111,"threads_started":1}
I20260812 03:07:17.107568 6620 heartbeater.cc:347] Connected to a master server at 127.5.96.190:33911
I20260812 03:07:17.107802 6620 heartbeater.cc:464] Registering TS with master...
I20260812 03:07:17.108240 6620 heartbeater.cc:511] Master 127.5.96.190:33911 requested a full tablet report, sending...
I20260812 03:07:17.109113 5950 ts_manager.cc:194] Registered new tserver with Master: 2cda0537314a4d138637894bfc9fb3c3 (127.5.96.131:44101)
I20260812 03:07:17.109630 5950 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.96.131:53599
I20260812 03:07:17.119396 5506 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:07:17.124585 6006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:17.121289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49326 (local address 127.5.96.190:33911)
0812 03:07:17.121379 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:17.121384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:17.121636 (+ 252us) server_negotiation.cc:407] Connection header received
0812 03:07:17.121744 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:17.121751 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:17.121789 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:17.121850 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:17.122066 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.122540 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.123271 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.123500 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.123563 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:17.124234 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:17.124237 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:17.124240 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:17.124253 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:17.124275 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:17.124285 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:17.124343 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:17.124399 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 03:07:17.124442 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 03:07:17.127029 5950 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49326:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:07:17.129976 5950 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:07:17.143292 6642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:17.137559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.130:35867 (local address 127.0.0.1:47578)
0812 03:07:17.138330 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:17.138378 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:17.138497 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:17.139117 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:17.139123 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:17.139146 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:17.139292 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:17.139301 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.140773 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.140776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:17.141892 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:17.141902 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.142791 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.142795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:17.142906 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:17.142942 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 03:07:17.143039 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 03:07:17.143153 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"mutex_wait_us":106,"thread_start_us":73,"threads_started":1}
I20260812 03:07:17.143334 6641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:17.137700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47578 (local address 127.5.96.130:35867)
0812 03:07:17.138629 (+ 929us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:17.138641 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:17.138666 (+ 25us) server_negotiation.cc:407] Connection header received
0812 03:07:17.138743 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:17.138748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:17.138962 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:17.139116 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:17.139424 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.140647 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.142018 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.142668 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.142765 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:17.142944 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:17.143085 (+ 141us) server_negotiation.cc:299] Negotiation successful
0812 03:07:17.143174 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":823,"thread_start_us":130,"threads_started":1}
I20260812 03:07:17.143877 6639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:17.137669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:41833 (local address 127.0.0.1:51038)
0812 03:07:17.138710 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:17.138723 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:17.138811 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:17.139285 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:17.139288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:17.139311 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:17.139407 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:17.139414 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.140806 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.140809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:17.142248 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:17.142259 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.143287 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.143291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:17.143373 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:17.143394 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 03:07:17.143623 (+ 229us) client_negotiation.cc:240] Negotiation successful
0812 03:07:17.143713 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":986,"mutex_wait_us":80,"spinlock_wait_cycles":768,"thread_start_us":97,"threads_started":1}
I20260812 03:07:17.143877 6634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:17.137911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:44101 (local address 127.0.0.1:57138)
0812 03:07:17.139156 (+ 1245us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:17.139168 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:17.139300 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:17.139853 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:17.139855 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:17.139869 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:17.139936 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:17.139939 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.140827 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.140830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:17.142248 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:17.142259 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.143486 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.143489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:17.143565 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:17.143585 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 03:07:17.143644 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 03:07:17.143722 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1185,"mutex_wait_us":167,"thread_start_us":88,"threads_started":1}
I20260812 03:07:17.144187 6644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:17.138172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57138 (local address 127.5.96.131:44101)
0812 03:07:17.139017 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:17.139027 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:17.139238 (+ 211us) server_negotiation.cc:407] Connection header received
0812 03:07:17.139438 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:17.139443 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:17.139596 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:17.139714 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:17.140042 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.140740 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.142434 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.143151 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.143264 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:17.143430 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:17.143941 (+ 511us) server_negotiation.cc:299] Negotiation successful
0812 03:07:17.144037 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":754,"thread_start_us":140,"threads_started":1}
I20260812 03:07:17.144733 6645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:17.138291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51038 (local address 127.5.96.129:41833)
0812 03:07:17.138809 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:17.138823 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:17.138865 (+ 42us) server_negotiation.cc:407] Connection header received
0812 03:07:17.138941 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:17.138946 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:17.139116 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:17.139299 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:17.139505 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.140646 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.142390 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.143151 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.144367 (+ 1216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:17.144505 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:17.144542 (+ 37us) server_negotiation.cc:299] Negotiation successful
0812 03:07:17.144604 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":114,"threads_started":1}
I20260812 03:07:17.146304 6333 tablet_service.cc:1467] Processing CreateTablet for tablet 360775e194924017aaab8166e32f6672 (DEFAULT_TABLE table=test-table [id=476728c5e9144b559bbbd36d66ab7f3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:07:17.147171 6333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 360775e194924017aaab8166e32f6672. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:17.146852 6519 tablet_service.cc:1467] Processing CreateTablet for tablet 360775e194924017aaab8166e32f6672 (DEFAULT_TABLE table=test-table [id=476728c5e9144b559bbbd36d66ab7f3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:07:17.147188 6130 tablet_service.cc:1467] Processing CreateTablet for tablet 360775e194924017aaab8166e32f6672 (DEFAULT_TABLE table=test-table [id=476728c5e9144b559bbbd36d66ab7f3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:07:17.147912 6519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 360775e194924017aaab8166e32f6672. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:17.148149 6130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 360775e194924017aaab8166e32f6672. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:17.152977 6646 tablet_bootstrap.cc:492] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57: Bootstrap starting.
I20260812 03:07:17.154664 6648 tablet_bootstrap.cc:492] T 360775e194924017aaab8166e32f6672 P 2cda0537314a4d138637894bfc9fb3c3: Bootstrap starting.
I20260812 03:07:17.155946 6649 tablet_bootstrap.cc:492] T 360775e194924017aaab8166e32f6672 P a2b9cd6122a9414d8764a6ba2ddd50f9: Bootstrap starting.
I20260812 03:07:17.156011 6646 tablet_bootstrap.cc:654] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:17.156769 6648 tablet_bootstrap.cc:654] T 360775e194924017aaab8166e32f6672 P 2cda0537314a4d138637894bfc9fb3c3: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:17.156847 6646 log.cc:824] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:17.157497 6648 log.cc:824] T 360775e194924017aaab8166e32f6672 P 2cda0537314a4d138637894bfc9fb3c3: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:17.157665 6649 tablet_bootstrap.cc:654] T 360775e194924017aaab8166e32f6672 P a2b9cd6122a9414d8764a6ba2ddd50f9: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:17.158411 6649 log.cc:824] T 360775e194924017aaab8166e32f6672 P a2b9cd6122a9414d8764a6ba2ddd50f9: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:17.158806 6646 tablet_bootstrap.cc:492] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57: No bootstrap required, opened a new log
I20260812 03:07:17.159021 6646 ts_tablet_manager.cc:1397] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:07:17.159355 6648 tablet_bootstrap.cc:492] T 360775e194924017aaab8166e32f6672 P 2cda0537314a4d138637894bfc9fb3c3: No bootstrap required, opened a new log
I20260812 03:07:17.159538 6648 ts_tablet_manager.cc:1397] T 360775e194924017aaab8166e32f6672 P 2cda0537314a4d138637894bfc9fb3c3: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 03:07:17.160204 6649 tablet_bootstrap.cc:492] T 360775e194924017aaab8166e32f6672 P a2b9cd6122a9414d8764a6ba2ddd50f9: No bootstrap required, opened a new log
I20260812 03:07:17.160375 6649 ts_tablet_manager.cc:1397] T 360775e194924017aaab8166e32f6672 P a2b9cd6122a9414d8764a6ba2ddd50f9: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260812 03:07:17.163218 6649 raft_consensus.cc:348] T 360775e194924017aaab8166e32f6672 P a2b9cd6122a9414d8764a6ba2ddd50f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22c87b1a8d4bb6b09646962a405c57" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35867 } } peers { permanent_uuid: "a2b9cd6122a9414d8764a6ba2ddd50f9" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 41833 } } peers { permanent_uuid: "2cda0537314a4d138637894bfc9fb3c3" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44101 } }
I20260812 03:07:17.163414 6649 raft_consensus.cc:374] T 360775e194924017aaab8166e32f6672 P a2b9cd6122a9414d8764a6ba2ddd50f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:17.163467 6649 raft_consensus.cc:729] T 360775e194924017aaab8166e32f6672 P a2b9cd6122a9414d8764a6ba2ddd50f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b9cd6122a9414d8764a6ba2ddd50f9, State: Initialized, Role: FOLLOWER
I20260812 03:07:17.163470 6646 raft_consensus.cc:348] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22c87b1a8d4bb6b09646962a405c57" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35867 } } peers { permanent_uuid: "a2b9cd6122a9414d8764a6ba2ddd50f9" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 41833 } } peers { permanent_uuid: "2cda0537314a4d138637894bfc9fb3c3" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44101 } }
I20260812 03:07:17.163470 6648 raft_consensus.cc:348] T 360775e194924017aaab8166e32f6672 P 2cda0537314a4d138637894bfc9fb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22c87b1a8d4bb6b09646962a405c57" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35867 } } peers { permanent_uuid: "a2b9cd6122a9414d8764a6ba2ddd50f9" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 41833 } } peers { permanent_uuid: "2cda0537314a4d138637894bfc9fb3c3" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44101 } }
I20260812 03:07:17.163728 6646 raft_consensus.cc:374] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:17.163728 6648 raft_consensus.cc:374] T 360775e194924017aaab8166e32f6672 P 2cda0537314a4d138637894bfc9fb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:17.163784 6646 raft_consensus.cc:729] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f22c87b1a8d4bb6b09646962a405c57, State: Initialized, Role: FOLLOWER
I20260812 03:07:17.163794 6648 raft_consensus.cc:729] T 360775e194924017aaab8166e32f6672 P 2cda0537314a4d138637894bfc9fb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cda0537314a4d138637894bfc9fb3c3, State: Initialized, Role: FOLLOWER
I20260812 03:07:17.163911 6649 consensus_queue.cc:260] T 360775e194924017aaab8166e32f6672 P a2b9cd6122a9414d8764a6ba2ddd50f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22c87b1a8d4bb6b09646962a405c57" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35867 } } peers { permanent_uuid: "a2b9cd6122a9414d8764a6ba2ddd50f9" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 41833 } } peers { permanent_uuid: "2cda0537314a4d138637894bfc9fb3c3" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44101 } }
I20260812 03:07:17.164196 6648 consensus_queue.cc:260] T 360775e194924017aaab8166e32f6672 P 2cda0537314a4d138637894bfc9fb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22c87b1a8d4bb6b09646962a405c57" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35867 } } peers { permanent_uuid: "a2b9cd6122a9414d8764a6ba2ddd50f9" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 41833 } } peers { permanent_uuid: "2cda0537314a4d138637894bfc9fb3c3" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44101 } }
I20260812 03:07:17.164197 6646 consensus_queue.cc:260] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22c87b1a8d4bb6b09646962a405c57" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35867 } } peers { permanent_uuid: "a2b9cd6122a9414d8764a6ba2ddd50f9" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 41833 } } peers { permanent_uuid: "2cda0537314a4d138637894bfc9fb3c3" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44101 } }
I20260812 03:07:17.164616 6230 heartbeater.cc:503] Master 127.5.96.190:33911 was elected leader, sending a full tablet report...
I20260812 03:07:17.164839 6649 ts_tablet_manager.cc:1428] T 360775e194924017aaab8166e32f6672 P a2b9cd6122a9414d8764a6ba2ddd50f9: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20260812 03:07:17.165114 6648 ts_tablet_manager.cc:1428] T 360775e194924017aaab8166e32f6672 P 2cda0537314a4d138637894bfc9fb3c3: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 03:07:17.165400 6620 heartbeater.cc:503] Master 127.5.96.190:33911 was elected leader, sending a full tablet report...
I20260812 03:07:17.165824 6425 heartbeater.cc:503] Master 127.5.96.190:33911 was elected leader, sending a full tablet report...
I20260812 03:07:17.167703 6646 ts_tablet_manager.cc:1428] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
W20260812 03:07:17.173811 6426 tablet.cc:2369] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:07:17.255131 6233 tablet.cc:2369] T 360775e194924017aaab8166e32f6672 P a2b9cd6122a9414d8764a6ba2ddd50f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:07:17.348243 6621 tablet.cc:2369] T 360775e194924017aaab8166e32f6672 P 2cda0537314a4d138637894bfc9fb3c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:07:17.459287 6655 raft_consensus.cc:482] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:07:17.459524 6655 raft_consensus.cc:504] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22c87b1a8d4bb6b09646962a405c57" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35867 } } peers { permanent_uuid: "a2b9cd6122a9414d8764a6ba2ddd50f9" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 41833 } } peers { permanent_uuid: "2cda0537314a4d138637894bfc9fb3c3" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44101 } }
I20260812 03:07:17.460728 6655 leader_election.cc:288] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b9cd6122a9414d8764a6ba2ddd50f9 (127.5.96.129:41833), 2cda0537314a4d138637894bfc9fb3c3 (127.5.96.131:44101)
I20260812 03:07:17.464486 6645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:17.461062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:37859 (local address 127.5.96.129:41833)
0812 03:07:17.461197 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:17.461203 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:17.461368 (+ 165us) server_negotiation.cc:407] Connection header received
0812 03:07:17.461567 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:17.461569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:17.461614 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:17.461679 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:17.461972 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.462483 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.463580 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.463994 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.464105 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:17.464157 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:17.464298 (+ 141us) server_negotiation.cc:299] Negotiation successful
0812 03:07:17.464361 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:07:17.464525 6661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:17.460952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:41833 (local address 127.5.96.130:37859)
0812 03:07:17.461284 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:17.461299 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:17.461398 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:17.461701 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:17.461704 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:17.461721 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:17.461863 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:17.461868 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.462591 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.462594 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:17.463462 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:17.463470 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.464105 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.464108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:17.464187 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:17.464209 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:07:17.464282 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 03:07:17.464350 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":77,"spinlock_wait_cycles":23808,"thread_start_us":99,"threads_started":1}
I20260812 03:07:17.464526 6644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:17.461145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.96.130:49791 (local address 127.5.96.131:44101)
0812 03:07:17.461263 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:17.461267 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:17.461480 (+ 213us) server_negotiation.cc:407] Connection header received
0812 03:07:17.461591 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:17.461594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:17.461641 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:17.461701 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:07:17.461995 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.462493 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.463573 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.464027 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.464134 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:17.464182 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:17.464343 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 03:07:17.464406 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:07:17.464547 6659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:17.460976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.131:44101 (local address 127.5.96.130:49791)
0812 03:07:17.461392 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:17.461406 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:17.461506 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:17.461717 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:17.461726 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:17.461768 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:07:17.461894 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:17.461899 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.462591 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.462594 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:17.463457 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:17.463466 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.464137 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.464141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:17.464234 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:17.464256 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:07:17.464320 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 03:07:17.464386 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":23,"spinlock_wait_cycles":1792,"thread_start_us":64,"threads_started":1}
I20260812 03:07:17.465108 6166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "360775e194924017aaab8166e32f6672" candidate_uuid: "3f22c87b1a8d4bb6b09646962a405c57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b9cd6122a9414d8764a6ba2ddd50f9" is_pre_election: true
I20260812 03:07:17.465118 6546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "360775e194924017aaab8166e32f6672" candidate_uuid: "3f22c87b1a8d4bb6b09646962a405c57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cda0537314a4d138637894bfc9fb3c3" is_pre_election: true
I20260812 03:07:17.465411 6546 raft_consensus.cc:2454] T 360775e194924017aaab8166e32f6672 P 2cda0537314a4d138637894bfc9fb3c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f22c87b1a8d4bb6b09646962a405c57 in term 0.
I20260812 03:07:17.465411 6166 raft_consensus.cc:2454] T 360775e194924017aaab8166e32f6672 P a2b9cd6122a9414d8764a6ba2ddd50f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f22c87b1a8d4bb6b09646962a405c57 in term 0.
I20260812 03:07:17.465909 6266 leader_election.cc:302] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2cda0537314a4d138637894bfc9fb3c3, 3f22c87b1a8d4bb6b09646962a405c57, a2b9cd6122a9414d8764a6ba2ddd50f9; no voters:
I20260812 03:07:17.466178 6655 raft_consensus.cc:2781] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:07:17.466264 6655 raft_consensus.cc:482] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:07:17.466316 6655 raft_consensus.cc:3037] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:17.467303 6655 raft_consensus.cc:504] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22c87b1a8d4bb6b09646962a405c57" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35867 } } peers { permanent_uuid: "a2b9cd6122a9414d8764a6ba2ddd50f9" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 41833 } } peers { permanent_uuid: "2cda0537314a4d138637894bfc9fb3c3" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44101 } }
I20260812 03:07:17.467602 6655 leader_election.cc:288] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57 [CANDIDATE]: Term 1 election: Requested vote from peers a2b9cd6122a9414d8764a6ba2ddd50f9 (127.5.96.129:41833), 2cda0537314a4d138637894bfc9fb3c3 (127.5.96.131:44101)
I20260812 03:07:17.467938 6166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "360775e194924017aaab8166e32f6672" candidate_uuid: "3f22c87b1a8d4bb6b09646962a405c57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b9cd6122a9414d8764a6ba2ddd50f9"
I20260812 03:07:17.467943 6546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "360775e194924017aaab8166e32f6672" candidate_uuid: "3f22c87b1a8d4bb6b09646962a405c57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cda0537314a4d138637894bfc9fb3c3"
I20260812 03:07:17.468036 6546 raft_consensus.cc:3037] T 360775e194924017aaab8166e32f6672 P 2cda0537314a4d138637894bfc9fb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:17.468040 6166 raft_consensus.cc:3037] T 360775e194924017aaab8166e32f6672 P a2b9cd6122a9414d8764a6ba2ddd50f9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:17.468981 6166 raft_consensus.cc:2454] T 360775e194924017aaab8166e32f6672 P a2b9cd6122a9414d8764a6ba2ddd50f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f22c87b1a8d4bb6b09646962a405c57 in term 1.
I20260812 03:07:17.469029 6546 raft_consensus.cc:2454] T 360775e194924017aaab8166e32f6672 P 2cda0537314a4d138637894bfc9fb3c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f22c87b1a8d4bb6b09646962a405c57 in term 1.
I20260812 03:07:17.469269 6266 leader_election.cc:302] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2cda0537314a4d138637894bfc9fb3c3, 3f22c87b1a8d4bb6b09646962a405c57, a2b9cd6122a9414d8764a6ba2ddd50f9; no voters:
I20260812 03:07:17.469435 6655 raft_consensus.cc:2781] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:17.469640 6655 raft_consensus.cc:686] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57 [term 1 LEADER]: Becoming Leader. State: Replica: 3f22c87b1a8d4bb6b09646962a405c57, State: Running, Role: LEADER
I20260812 03:07:17.470086 6655 consensus_queue.cc:237] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22c87b1a8d4bb6b09646962a405c57" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35867 } } peers { permanent_uuid: "a2b9cd6122a9414d8764a6ba2ddd50f9" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 41833 } } peers { permanent_uuid: "2cda0537314a4d138637894bfc9fb3c3" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44101 } }
I20260812 03:07:17.473183 5950 catalog_manager.cc:5495] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f22c87b1a8d4bb6b09646962a405c57 (127.5.96.130). New cstate: current_term: 1 leader_uuid: "3f22c87b1a8d4bb6b09646962a405c57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f22c87b1a8d4bb6b09646962a405c57" member_type: VOTER last_known_addr { host: "127.5.96.130" port: 35867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b9cd6122a9414d8764a6ba2ddd50f9" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 41833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cda0537314a4d138637894bfc9fb3c3" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44101 } health_report { overall_health: UNKNOWN } } }
I20260812 03:07:17.499816 6641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:17.496946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47590 (local address 127.5.96.130:35867)
0812 03:07:17.497044 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:17.497048 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:17.497061 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:07:17.497185 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:17.497187 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:17.497234 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:17.497325 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:07:17.497495 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.497949 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.498708 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:17.498904 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:17.498965 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:17.499167 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:07:17.499536 (+ 369us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:07:17.499617 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:17.499643 (+ 26us) server_negotiation.cc:299] Negotiation successful
0812 03:07:17.499712 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 03:07:17.505316 6166 raft_consensus.cc:1261] T 360775e194924017aaab8166e32f6672 P a2b9cd6122a9414d8764a6ba2ddd50f9 [term 1 FOLLOWER]: Refusing update from remote peer 3f22c87b1a8d4bb6b09646962a405c57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:07:17.505353 6546 raft_consensus.cc:1261] T 360775e194924017aaab8166e32f6672 P 2cda0537314a4d138637894bfc9fb3c3 [term 1 FOLLOWER]: Refusing update from remote peer 3f22c87b1a8d4bb6b09646962a405c57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:07:17.505945 6655 consensus_queue.cc:1035] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b9cd6122a9414d8764a6ba2ddd50f9" member_type: VOTER last_known_addr { host: "127.5.96.129" port: 41833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:07:17.506136 6662 consensus_queue.cc:1035] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cda0537314a4d138637894bfc9fb3c3" member_type: VOTER last_known_addr { host: "127.5.96.131" port: 44101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:07:17.516888 6668 mvcc.cc:205] Tried to move back new op lower bound from 7317520537615462400 to 7317520537479827456. Current Snapshot: MvccSnapshot[applied={T|T < 7317520537615462400}]
I20260812 03:07:17.522957 5950 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:07:17.523087 5950 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:07:17.525274 5950 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a88afbe3635d4c84809e58dec3f4d759: Sending DeleteTablet for 3 replicas of tablet 360775e194924017aaab8166e32f6672
I20260812 03:07:17.525959 6130 tablet_service.cc:1514] Processing DeleteTablet for tablet 360775e194924017aaab8166e32f6672 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:07:17 UTC) from {username='slave'} at 127.0.0.1:51038
I20260812 03:07:17.525940 6519 tablet_service.cc:1514] Processing DeleteTablet for tablet 360775e194924017aaab8166e32f6672 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:07:17 UTC) from {username='slave'} at 127.0.0.1:57138
I20260812 03:07:17.525977 6333 tablet_service.cc:1514] Processing DeleteTablet for tablet 360775e194924017aaab8166e32f6672 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:07:17 UTC) from {username='slave'} at 127.0.0.1:47578
I20260812 03:07:17.526532 6677 tablet_replica.cc:330] T 360775e194924017aaab8166e32f6672 P 2cda0537314a4d138637894bfc9fb3c3: stopping tablet replica
I20260812 03:07:17.526762 6678 tablet_replica.cc:330] T 360775e194924017aaab8166e32f6672 P a2b9cd6122a9414d8764a6ba2ddd50f9: stopping tablet replica
I20260812 03:07:17.526855 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 6040
I20260812 03:07:17.527025 6680 tablet_replica.cc:330] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57: stopping tablet replica
I20260812 03:07:17.533948 6677 raft_consensus.cc:2229] T 360775e194924017aaab8166e32f6672 P 2cda0537314a4d138637894bfc9fb3c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:07:17.534299 6680 raft_consensus.cc:2229] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:07:17.534677 6680 raft_consensus.cc:2258] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:07:17.535022 6677 raft_consensus.cc:2258] T 360775e194924017aaab8166e32f6672 P 2cda0537314a4d138637894bfc9fb3c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:07:17.536495 6680 ts_tablet_manager.cc:1905] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:07:17.539147 6680 ts_tablet_manager.cc:1918] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:07:17.539297 6680 log.cc:1196] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-1/wal/wals/360775e194924017aaab8166e32f6672
I20260812 03:07:17.539692 6680 ts_tablet_manager.cc:1939] T 360775e194924017aaab8166e32f6672 P 3f22c87b1a8d4bb6b09646962a405c57: Deleting consensus metadata
I20260812 03:07:17.540843 5924 catalog_manager.cc:4841] TS 3f22c87b1a8d4bb6b09646962a405c57 (127.5.96.130:35867): tablet 360775e194924017aaab8166e32f6672 (table test-table [id=476728c5e9144b559bbbd36d66ab7f3d]) successfully deleted
I20260812 03:07:17.540916 6677 ts_tablet_manager.cc:1905] T 360775e194924017aaab8166e32f6672 P 2cda0537314a4d138637894bfc9fb3c3: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 03:07:17.542342 5927 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.96.129:41833 (error 108)
I20260812 03:07:17.543244 6639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:17.542981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.129:41833 (local address 127.0.0.1:51042)
0812 03:07:17.543092 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:17.543158 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.129:41833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 03:07:17.543375 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 6235
I20260812 03:07:17.543447 6677 ts_tablet_manager.cc:1918] T 360775e194924017aaab8166e32f6672 P 2cda0537314a4d138637894bfc9fb3c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:07:17.545858 6677 log.cc:1196] T 360775e194924017aaab8166e32f6672 P 2cda0537314a4d138637894bfc9fb3c3: Deleting WAL directory at /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786503790637001-5506-0/minicluster-data/ts-2/wal/wals/360775e194924017aaab8166e32f6672
W20260812 03:07:17.546010 5927 catalog_manager.cc:4584] TS a2b9cd6122a9414d8764a6ba2ddd50f9 (127.5.96.129:41833): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 360775e194924017aaab8166e32f6672: Network error: Client connection negotiation failed: client connection to 127.5.96.129:41833: connect: Connection refused (error 111)
I20260812 03:07:17.546169 6677 ts_tablet_manager.cc:1939] T 360775e194924017aaab8166e32f6672 P 2cda0537314a4d138637894bfc9fb3c3: Deleting consensus metadata
I20260812 03:07:17.547144 5925 catalog_manager.cc:4841] TS 2cda0537314a4d138637894bfc9fb3c3 (127.5.96.131:44101): tablet 360775e194924017aaab8166e32f6672 (table test-table [id=476728c5e9144b559bbbd36d66ab7f3d]) successfully deleted
I20260812 03:07:17.551081 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 6430
I20260812 03:07:17.564049 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 5902
2026-08-12T03:07:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1256 ms)
[----------] 6 tests from EncryptionPolicyTest (10689 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:07:17 dist-test-slave-6bbx krb5kdc[6686](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:07:17 dist-test-slave-6bbx krb5kdc[6686](info): set up 2 sockets
Aug 12 03:07:17 dist-test-slave-6bbx krb5kdc[6686](info): commencing operation
krb5kdc: starting...
W20260812 03:07:18.715353 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:07:18 dist-test-slave-6bbx krb5kdc[6686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:07:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:07:18.824585 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39835
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39835
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:18.932312 6701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:18.932531 6701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:18.932586 6701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:18.936430 6701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:18.936487 6701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:18.936514 6701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:18.936543 6701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:18.936566 6701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:18.941574 6701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39835
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:18.942479 6701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:18.943396 6701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:18.951133 6716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:18.951149 6713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:18.951238 6701 server_base.cc:1034] running on GCE node
W20260812 03:07:18.951184 6711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:18.951911 6701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:18.952872 6701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:18.954021 6701 hybrid_clock.cc:648] HybridClock initialized: now 1786504038954007 us; error 23 us; skew 500 ppm
Aug 12 03:07:18 dist-test-slave-6bbx krb5kdc[6686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504038, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:18.956545 6701 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:07:18.957329 6701 webserver.cc:460] Webserver started at http://127.0.0.1:44023/ using document root <none> and password file <none>
I20260812 03:07:18.957808 6701 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:18.957872 6701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:18.958045 6701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:18.959959 6701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "c000db7513f745f0b483f3b660b73d1e"
format_stamp: "Formatted at 2026-08-12 03:07:18 on dist-test-slave-6bbx"
server_key: "90d3e60a05ad792b4a2a0c8547248f51"
server_key_iv: "b1b35bc3af606bd69934f99953d9b13c"
server_key_version: "encryptionkey@0"
I20260812 03:07:18.960474 6701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "c000db7513f745f0b483f3b660b73d1e"
format_stamp: "Formatted at 2026-08-12 03:07:18 on dist-test-slave-6bbx"
server_key: "90d3e60a05ad792b4a2a0c8547248f51"
server_key_iv: "b1b35bc3af606bd69934f99953d9b13c"
server_key_version: "encryptionkey@0"
I20260812 03:07:18.963863 6701 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 03:07:18.966058 6725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:18.967142 6701 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:07:18.967257 6701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "c000db7513f745f0b483f3b660b73d1e"
format_stamp: "Formatted at 2026-08-12 03:07:18 on dist-test-slave-6bbx"
server_key: "90d3e60a05ad792b4a2a0c8547248f51"
server_key_iv: "b1b35bc3af606bd69934f99953d9b13c"
server_key_version: "encryptionkey@0"
I20260812 03:07:18.967347 6701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:18.987637 6701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:18.990216 6701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:18.990381 6701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:18.996814 6701 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39835
I20260812 03:07:18.996829 6807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39835 every 8 connection(s)
I20260812 03:07:18.997638 6701 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:18.999495 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 6701
I20260812 03:07:18.999603 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:07:18.999878 5506 external_mini_cluster.cc:1428] Setting key baf9cc202f875301600026af6d0ea57b
I20260812 03:07:19.000309 6808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:19.005954 6808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c000db7513f745f0b483f3b660b73d1e: Bootstrap starting.
Aug 12 03:07:19 dist-test-slave-6bbx krb5kdc[6686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 03:07:19.008167 6808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c000db7513f745f0b483f3b660b73d1e: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:19.008908 6808 log.cc:824] T 00000000000000000000000000000000 P c000db7513f745f0b483f3b660b73d1e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:19.010667 6808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c000db7513f745f0b483f3b660b73d1e: No bootstrap required, opened a new log
I20260812 03:07:19.012719 6813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:19.001141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54709 (local address 127.0.0.1:39835)
0812 03:07:19.001575 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:19.001590 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:19.001623 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:07:19.002255 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:19.002272 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:19.002573 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:19.002862 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:19.003634 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:19.004615 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:19.005443 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:19.005898 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:19.006036 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:19.008441 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:19.008446 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:19.008458 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:19.008483 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:19.010080 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:19.010561 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:19.010565 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:19.010570 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:19.010681 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:19.010997 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:19.010999 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:19.011001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:19.011355 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:19.011553 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:19.011883 (+ 330us) server_negotiation.cc:299] Negotiation successful
0812 03:07:19.012076 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":142,"threads_started":1}
I20260812 03:07:19.014317 6808 raft_consensus.cc:348] T 00000000000000000000000000000000 P c000db7513f745f0b483f3b660b73d1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c000db7513f745f0b483f3b660b73d1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39835 } }
I20260812 03:07:19.014616 6808 raft_consensus.cc:374] T 00000000000000000000000000000000 P c000db7513f745f0b483f3b660b73d1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:19.014685 6808 raft_consensus.cc:729] T 00000000000000000000000000000000 P c000db7513f745f0b483f3b660b73d1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c000db7513f745f0b483f3b660b73d1e, State: Initialized, Role: FOLLOWER
I20260812 03:07:19.015097 6808 consensus_queue.cc:260] T 00000000000000000000000000000000 P c000db7513f745f0b483f3b660b73d1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c000db7513f745f0b483f3b660b73d1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39835 } }
I20260812 03:07:19.015264 6808 raft_consensus.cc:388] T 00000000000000000000000000000000 P c000db7513f745f0b483f3b660b73d1e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:07:19.015368 6808 raft_consensus.cc:482] T 00000000000000000000000000000000 P c000db7513f745f0b483f3b660b73d1e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:07:19.015476 6808 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c000db7513f745f0b483f3b660b73d1e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:19.015684 5506 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:07:19.016397 6808 raft_consensus.cc:504] T 00000000000000000000000000000000 P c000db7513f745f0b483f3b660b73d1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c000db7513f745f0b483f3b660b73d1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39835 } }
I20260812 03:07:19.016680 6808 leader_election.cc:302] T 00000000000000000000000000000000 P c000db7513f745f0b483f3b660b73d1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c000db7513f745f0b483f3b660b73d1e; no voters:
I20260812 03:07:19.017253 6808 leader_election.cc:288] T 00000000000000000000000000000000 P c000db7513f745f0b483f3b660b73d1e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:07:19.017431 6818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c000db7513f745f0b483f3b660b73d1e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:19.017604 6818 raft_consensus.cc:686] T 00000000000000000000000000000000 P c000db7513f745f0b483f3b660b73d1e [term 1 LEADER]: Becoming Leader. State: Replica: c000db7513f745f0b483f3b660b73d1e, State: Running, Role: LEADER
I20260812 03:07:19.017872 6818 consensus_queue.cc:237] T 00000000000000000000000000000000 P c000db7513f745f0b483f3b660b73d1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c000db7513f745f0b483f3b660b73d1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39835 } }
I20260812 03:07:19.018452 6808 sys_catalog.cc:564] T 00000000000000000000000000000000 P c000db7513f745f0b483f3b660b73d1e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:19.018998 6818 sys_catalog.cc:455] T 00000000000000000000000000000000 P c000db7513f745f0b483f3b660b73d1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c000db7513f745f0b483f3b660b73d1e. Latest consensus state: current_term: 1 leader_uuid: "c000db7513f745f0b483f3b660b73d1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c000db7513f745f0b483f3b660b73d1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39835 } } }
I20260812 03:07:19.019168 6818 sys_catalog.cc:458] T 00000000000000000000000000000000 P c000db7513f745f0b483f3b660b73d1e [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:19.019501 6823 sys_catalog.cc:455] T 00000000000000000000000000000000 P c000db7513f745f0b483f3b660b73d1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c000db7513f745f0b483f3b660b73d1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c000db7513f745f0b483f3b660b73d1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39835 } } }
I20260812 03:07:19.019702 6823 sys_catalog.cc:458] T 00000000000000000000000000000000 P c000db7513f745f0b483f3b660b73d1e [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:19.019573 6828 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:19.022737 6828 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:19.028061 6813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:19.019951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57209 (local address 127.0.0.1:39835)
0812 03:07:19.020057 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:19.020061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:19.020489 (+ 428us) server_negotiation.cc:407] Connection header received
0812 03:07:19.020612 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:19.020615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:19.020666 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:19.020753 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:19.021470 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:19.022234 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:19.023276 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:19.023579 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:19.023652 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:19.025255 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:19.025258 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:19.025260 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:19.025281 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:19.026453 (+ 1172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:19.026899 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:19.026902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:19.026904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:19.026975 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:19.027274 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:19.027278 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:19.027280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:19.027432 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:19.027556 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:19.027841 (+ 285us) server_negotiation.cc:299] Negotiation successful
0812 03:07:19.027924 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 03:07:19.029304 6828 catalog_manager.cc:1347] Generated new cluster ID: ed6a6df73a8245dd834d3d759eddce3f
I20260812 03:07:19.029366 6828 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:19.056922 6828 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:19.057739 6828 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:19.064653 6828 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c000db7513f745f0b483f3b660b73d1e: Generated new TSK 0
I20260812 03:07:19.065219 6828 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:07:19.069945 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 6701
W20260812 03:07:19.087116 6684 connection.cc:535] client connection to 127.0.0.1:39835 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39835): Connection reset by peer (error 104)
2026-08-12T03:07:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1511 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:07:19 dist-test-slave-6bbx krb5kdc[6853](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:07:19 dist-test-slave-6bbx krb5kdc[6853](info): set up 2 sockets
Aug 12 03:07:19 dist-test-slave-6bbx krb5kdc[6853](info): commencing operation
krb5kdc: starting...
W20260812 03:07:20.225059 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:07:20 dist-test-slave-6bbx krb5kdc[6853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:07:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:07:20.334437 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33783
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33783
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:20.440026 6873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:20.440258 6873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:20.440315 6873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:20.444216 6873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:20.444270 6873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:20.444296 6873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:20.444316 6873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:20.444341 6873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:20.449314 6873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33783
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:20.450224 6873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:20.451136 6873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:20.458956 6885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:20.459069 6873 server_base.cc:1034] running on GCE node
W20260812 03:07:20.458954 6881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:20.459281 6882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:20.459852 6873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:20.460841 6873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:20.462005 6873 hybrid_clock.cc:648] HybridClock initialized: now 1786504040461985 us; error 28 us; skew 500 ppm
Aug 12 03:07:20 dist-test-slave-6bbx krb5kdc[6853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504040, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:20.470671 6873 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:07:20.471480 6873 webserver.cc:460] Webserver started at http://127.0.0.1:45873/ using document root <none> and password file <none>
I20260812 03:07:20.472007 6873 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:20.472078 6873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:20.472256 6873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:20.474110 6873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "fc6b393215ef440a99fba5a07335f57d"
format_stamp: "Formatted at 2026-08-12 03:07:20 on dist-test-slave-6bbx"
server_key: "5352c0d57e49b0adbf6e7d77324b66fa"
server_key_iv: "d31b2e4e7cc18c25ed8b288dd35cf392"
server_key_version: "encryptionkey@0"
I20260812 03:07:20.474593 6873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "fc6b393215ef440a99fba5a07335f57d"
format_stamp: "Formatted at 2026-08-12 03:07:20 on dist-test-slave-6bbx"
server_key: "5352c0d57e49b0adbf6e7d77324b66fa"
server_key_iv: "d31b2e4e7cc18c25ed8b288dd35cf392"
server_key_version: "encryptionkey@0"
I20260812 03:07:20.477998 6873 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260812 03:07:20.480227 6893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:20.481328 6873 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:07:20.481436 6873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "fc6b393215ef440a99fba5a07335f57d"
format_stamp: "Formatted at 2026-08-12 03:07:20 on dist-test-slave-6bbx"
server_key: "5352c0d57e49b0adbf6e7d77324b66fa"
server_key_iv: "d31b2e4e7cc18c25ed8b288dd35cf392"
server_key_version: "encryptionkey@0"
I20260812 03:07:20.481515 6873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:20.497962 6873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:20.500404 6873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:20.500540 6873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:20.507319 6873 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33783
I20260812 03:07:20.507329 6972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33783 every 8 connection(s)
I20260812 03:07:20.508213 6873 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:20.509294 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 6873
I20260812 03:07:20.509438 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:07:20.509757 5506 external_mini_cluster.cc:1428] Setting key 7978eaff54639a879544575d18614cd0
I20260812 03:07:20.511264 6974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:20.516534 6974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc6b393215ef440a99fba5a07335f57d: Bootstrap starting.
I20260812 03:07:20.517865 6979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:20.511051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46219 (local address 127.0.0.1:33783)
0812 03:07:20.511485 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:20.511499 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:20.511532 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:07:20.512186 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:20.512203 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:20.512512 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:20.512797 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:20.513165 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:20.514090 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:20.514942 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:20.515385 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:20.515524 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:20.516352 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:20.516357 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:20.516368 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:20.516736 (+ 368us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:20.516786 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:20.516808 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:20.516916 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:20.517109 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 03:07:20.517232 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":137,"threads_started":1}
I20260812 03:07:20.518779 6974 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc6b393215ef440a99fba5a07335f57d: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:20.519603 6974 log.cc:824] T 00000000000000000000000000000000 P fc6b393215ef440a99fba5a07335f57d: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:20.520795 5506 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:07:20.521415 6974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc6b393215ef440a99fba5a07335f57d: No bootstrap required, opened a new log
I20260812 03:07:20.525393 6974 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc6b393215ef440a99fba5a07335f57d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6b393215ef440a99fba5a07335f57d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33783 } }
I20260812 03:07:20.525658 6974 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc6b393215ef440a99fba5a07335f57d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:20.525767 6974 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc6b393215ef440a99fba5a07335f57d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc6b393215ef440a99fba5a07335f57d, State: Initialized, Role: FOLLOWER
I20260812 03:07:20.526206 6974 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc6b393215ef440a99fba5a07335f57d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6b393215ef440a99fba5a07335f57d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33783 } }
I20260812 03:07:20.526367 6974 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc6b393215ef440a99fba5a07335f57d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:07:20.526463 6974 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc6b393215ef440a99fba5a07335f57d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:07:20.526489 6979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:20.522618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51539 (local address 127.0.0.1:33783)
0812 03:07:20.522719 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:20.522723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:20.522956 (+ 233us) server_negotiation.cc:407] Connection header received
0812 03:07:20.523065 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:20.523067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:20.523101 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:20.523160 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:20.523434 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:20.523933 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:20.525018 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:20.525218 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:20.525286 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:20.526052 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:20.526056 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:20.526058 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:20.526071 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:20.526096 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:20.526107 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:20.526173 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:20.526307 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 03:07:20.526335 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 03:07:20.526566 6974 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc6b393215ef440a99fba5a07335f57d [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:20.527359 6974 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc6b393215ef440a99fba5a07335f57d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6b393215ef440a99fba5a07335f57d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33783 } }
I20260812 03:07:20.527743 6974 leader_election.cc:302] T 00000000000000000000000000000000 P fc6b393215ef440a99fba5a07335f57d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc6b393215ef440a99fba5a07335f57d; no voters:
I20260812 03:07:20.528020 6974 leader_election.cc:288] T 00000000000000000000000000000000 P fc6b393215ef440a99fba5a07335f57d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:07:20.528129 6987 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc6b393215ef440a99fba5a07335f57d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:20.528349 6987 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc6b393215ef440a99fba5a07335f57d [term 1 LEADER]: Becoming Leader. State: Replica: fc6b393215ef440a99fba5a07335f57d, State: Running, Role: LEADER
I20260812 03:07:20.528638 6987 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc6b393215ef440a99fba5a07335f57d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6b393215ef440a99fba5a07335f57d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33783 } }
I20260812 03:07:20.529179 6974 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc6b393215ef440a99fba5a07335f57d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:20.530184 6995 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc6b393215ef440a99fba5a07335f57d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc6b393215ef440a99fba5a07335f57d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6b393215ef440a99fba5a07335f57d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33783 } } }
I20260812 03:07:20.530328 6995 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc6b393215ef440a99fba5a07335f57d [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:20.530701 7005 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:20.530679 6996 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc6b393215ef440a99fba5a07335f57d [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc6b393215ef440a99fba5a07335f57d. Latest consensus state: current_term: 1 leader_uuid: "fc6b393215ef440a99fba5a07335f57d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6b393215ef440a99fba5a07335f57d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33783 } } }
I20260812 03:07:20.531253 6996 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc6b393215ef440a99fba5a07335f57d [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:20.533682 7005 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:20.538621 7005 catalog_manager.cc:1347] Generated new cluster ID: afe265314bbb45fa82bf99a0c332318f
I20260812 03:07:20.538673 7005 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:20.555922 7005 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:20.557237 7005 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:20.566398 7005 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc6b393215ef440a99fba5a07335f57d: Generated new TSK 0
I20260812 03:07:20.567082 7005 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:07:20.576517 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 6873
W20260812 03:07:20.589179 6850 connection.cc:535] client connection to 127.0.0.1:33783 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33783): Connection reset by peer (error 104)
2026-08-12T03:07:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1501 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:07:20 dist-test-slave-6bbx krb5kdc[7023](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:07:20 dist-test-slave-6bbx krb5kdc[7023](info): set up 2 sockets
Aug 12 03:07:20 dist-test-slave-6bbx krb5kdc[7023](info): commencing operation
krb5kdc: starting...
W20260812 03:07:21.721838 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:07:21 dist-test-slave-6bbx krb5kdc[7023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:07:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:07:21.832429 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41861
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:21.940723 7043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:21.940953 7043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:21.941007 7043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:21.944878 7043 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:21.944931 7043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:21.944955 7043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:21.944976 7043 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:21.945004 7043 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:21.950034 7043 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41861
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:21.950933 7043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:21.951901 7043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:07:21.959770 7043 server_base.cc:1034] running on GCE node
W20260812 03:07:21.959695 7054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:21.959697 7052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:21.959757 7060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:21.960438 7043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:21.961445 7043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:21.962605 7043 hybrid_clock.cc:648] HybridClock initialized: now 1786504041962582 us; error 33 us; skew 500 ppm
Aug 12 03:07:21 dist-test-slave-6bbx krb5kdc[7023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504041, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:21.965181 7043 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:07:21.965987 7043 webserver.cc:460] Webserver started at http://127.0.0.1:36107/ using document root <none> and password file <none>
I20260812 03:07:21.966482 7043 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:21.966545 7043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:21.966727 7043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:21.968637 7043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "cfd93a68eea9434bac74b57adb8e9b95"
format_stamp: "Formatted at 2026-08-12 03:07:21 on dist-test-slave-6bbx"
server_key: "da22d16617070f2b8d07d57720ed56f2"
server_key_iv: "c391b72288c69461847849dfb286f889"
server_key_version: "encryptionkey@0"
I20260812 03:07:21.969167 7043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "cfd93a68eea9434bac74b57adb8e9b95"
format_stamp: "Formatted at 2026-08-12 03:07:21 on dist-test-slave-6bbx"
server_key: "da22d16617070f2b8d07d57720ed56f2"
server_key_iv: "c391b72288c69461847849dfb286f889"
server_key_version: "encryptionkey@0"
I20260812 03:07:21.972560 7043 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:07:21.974740 7074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:21.975862 7043 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:07:21.975971 7043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "cfd93a68eea9434bac74b57adb8e9b95"
format_stamp: "Formatted at 2026-08-12 03:07:21 on dist-test-slave-6bbx"
server_key: "da22d16617070f2b8d07d57720ed56f2"
server_key_iv: "c391b72288c69461847849dfb286f889"
server_key_version: "encryptionkey@0"
I20260812 03:07:21.976063 7043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:21.980311 7043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:21.982818 7043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:21.982976 7043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:21.989562 7043 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41861
I20260812 03:07:21.989574 7157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41861 every 8 connection(s)
I20260812 03:07:21.990396 7043 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:21.992985 7158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:21.997249 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 7043
I20260812 03:07:21.997380 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:07:21.997689 5506 external_mini_cluster.cc:1428] Setting key f008fb4c3d2d2501a72dff5d0ac77cd8
I20260812 03:07:21.998382 7158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfd93a68eea9434bac74b57adb8e9b95: Bootstrap starting.
I20260812 03:07:22.000988 7158 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfd93a68eea9434bac74b57adb8e9b95: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:22.001857 7158 log.cc:824] T 00000000000000000000000000000000 P cfd93a68eea9434bac74b57adb8e9b95: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:22.002794 7164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:21.998915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39377 (local address 127.0.0.1:41861)
0812 03:07:21.999311 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:21.999329 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:21.999367 (+ 38us) server_negotiation.cc:407] Connection header received
0812 03:07:22.000200 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:22.000222 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:22.000551 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:22.000890 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:22.001546 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:22.001552 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:22.001566 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:22.001576 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:22.001636 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:22.001664 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:22.001788 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:22.001854 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 03:07:22.002000 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":98,"threads_started":1}
I20260812 03:07:22.003949 7158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfd93a68eea9434bac74b57adb8e9b95: No bootstrap required, opened a new log
I20260812 03:07:22.005421 5506 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:07:22.007602 7158 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfd93a68eea9434bac74b57adb8e9b95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd93a68eea9434bac74b57adb8e9b95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41861 } }
I20260812 03:07:22.007839 7158 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfd93a68eea9434bac74b57adb8e9b95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:22.007898 7158 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfd93a68eea9434bac74b57adb8e9b95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfd93a68eea9434bac74b57adb8e9b95, State: Initialized, Role: FOLLOWER
I20260812 03:07:22.008298 7158 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfd93a68eea9434bac74b57adb8e9b95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd93a68eea9434bac74b57adb8e9b95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41861 } }
I20260812 03:07:22.008425 7158 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfd93a68eea9434bac74b57adb8e9b95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:07:22.008491 7158 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfd93a68eea9434bac74b57adb8e9b95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:07:22.008586 7158 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfd93a68eea9434bac74b57adb8e9b95 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:22.008783 7164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:22.007309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36845 (local address 127.0.0.1:41861)
0812 03:07:22.007407 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:22.007411 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:22.007589 (+ 178us) server_negotiation.cc:407] Connection header received
0812 03:07:22.007729 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:22.007732 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:22.007764 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:22.007826 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:22.008447 (+ 621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:22.008450 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:22.008453 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:22.008464 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:22.008491 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:22.008503 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:22.008575 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:22.008644 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 03:07:22.008679 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 03:07:22.009516 7158 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfd93a68eea9434bac74b57adb8e9b95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd93a68eea9434bac74b57adb8e9b95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41861 } }
I20260812 03:07:22.009837 7158 leader_election.cc:302] T 00000000000000000000000000000000 P cfd93a68eea9434bac74b57adb8e9b95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfd93a68eea9434bac74b57adb8e9b95; no voters:
I20260812 03:07:22.010090 7158 leader_election.cc:288] T 00000000000000000000000000000000 P cfd93a68eea9434bac74b57adb8e9b95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:07:22.010186 7174 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfd93a68eea9434bac74b57adb8e9b95 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:22.010396 7174 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfd93a68eea9434bac74b57adb8e9b95 [term 1 LEADER]: Becoming Leader. State: Replica: cfd93a68eea9434bac74b57adb8e9b95, State: Running, Role: LEADER
I20260812 03:07:22.010747 7174 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfd93a68eea9434bac74b57adb8e9b95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd93a68eea9434bac74b57adb8e9b95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41861 } }
I20260812 03:07:22.011157 7158 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfd93a68eea9434bac74b57adb8e9b95 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:22.012333 7176 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfd93a68eea9434bac74b57adb8e9b95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfd93a68eea9434bac74b57adb8e9b95. Latest consensus state: current_term: 1 leader_uuid: "cfd93a68eea9434bac74b57adb8e9b95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd93a68eea9434bac74b57adb8e9b95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41861 } } }
I20260812 03:07:22.012333 7175 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfd93a68eea9434bac74b57adb8e9b95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfd93a68eea9434bac74b57adb8e9b95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd93a68eea9434bac74b57adb8e9b95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41861 } } }
I20260812 03:07:22.012509 7176 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfd93a68eea9434bac74b57adb8e9b95 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:22.012521 7175 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfd93a68eea9434bac74b57adb8e9b95 [sys.catalog]: This master's current role is: LEADER
W20260812 03:07:22.015595 7197 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cfd93a68eea9434bac74b57adb8e9b95: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:07:22.015730 7197 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:07:22.015820 7190 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:22.016497 7190 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:22.021504 7190 catalog_manager.cc:1347] Generated new cluster ID: 5adc909871d74955acb1c8164f92e5e7
I20260812 03:07:22.021564 7190 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:22.031335 7190 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:22.031615 7190 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:22.042099 7190 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfd93a68eea9434bac74b57adb8e9b95: Generated new TSK 0
I20260812 03:07:22.042755 7190 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:07:22.049115 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 7043
2026-08-12T03:07:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1470 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:07:22 dist-test-slave-6bbx krb5kdc[7205](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:07:22 dist-test-slave-6bbx krb5kdc[7205](info): set up 2 sockets
Aug 12 03:07:22 dist-test-slave-6bbx krb5kdc[7205](info): commencing operation
krb5kdc: starting...
W20260812 03:07:23.211448 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:07:23 dist-test-slave-6bbx krb5kdc[7205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:07:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:07:23.320544 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43491
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:23.426347 7231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:23.426580 7231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:23.426617 7231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:23.430442 7231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:23.430493 7231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:23.430511 7231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:23.430528 7231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:23.430544 7231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:23.435487 7231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:23.436416 7231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:23.437317 7231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:23.445292 7243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:23.445292 7236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:23.445346 7239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:23.445626 7231 server_base.cc:1034] running on GCE node
I20260812 03:07:23.446143 7231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:23.447140 7231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:23.448293 7231 hybrid_clock.cc:648] HybridClock initialized: now 1786504043448276 us; error 31 us; skew 500 ppm
Aug 12 03:07:23 dist-test-slave-6bbx krb5kdc[7205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504043, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:23.450837 7231 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:07:23.451613 7231 webserver.cc:460] Webserver started at http://127.0.0.1:33345/ using document root <none> and password file <none>
I20260812 03:07:23.452121 7231 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:23.452181 7231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:23.452348 7231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:23.454211 7231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "03d9ee425bb54b36ac818e1945008403"
format_stamp: "Formatted at 2026-08-12 03:07:23 on dist-test-slave-6bbx"
server_key: "120eb3613a0d6793280f2ee87b72b16b"
server_key_iv: "0d66e0d63ed8606213c1a6806d0a903e"
server_key_version: "encryptionkey@0"
I20260812 03:07:23.454708 7231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "03d9ee425bb54b36ac818e1945008403"
format_stamp: "Formatted at 2026-08-12 03:07:23 on dist-test-slave-6bbx"
server_key: "120eb3613a0d6793280f2ee87b72b16b"
server_key_iv: "0d66e0d63ed8606213c1a6806d0a903e"
server_key_version: "encryptionkey@0"
I20260812 03:07:23.458103 7231 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 03:07:23.460393 7254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:23.461483 7231 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:07:23.461580 7231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "03d9ee425bb54b36ac818e1945008403"
format_stamp: "Formatted at 2026-08-12 03:07:23 on dist-test-slave-6bbx"
server_key: "120eb3613a0d6793280f2ee87b72b16b"
server_key_iv: "0d66e0d63ed8606213c1a6806d0a903e"
server_key_version: "encryptionkey@0"
I20260812 03:07:23.461658 7231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:23.477650 7231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:23.480427 7231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:23.480564 7231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:23.487227 7231 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43491
I20260812 03:07:23.487242 7334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43491 every 8 connection(s)
I20260812 03:07:23.488138 7231 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:23.490679 7335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:23.495333 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 7231
I20260812 03:07:23.495457 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:07:23.495772 5506 external_mini_cluster.cc:1428] Setting key 3824994b10274db9022504c251589b41
I20260812 03:07:23.496029 7335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03d9ee425bb54b36ac818e1945008403: Bootstrap starting.
I20260812 03:07:23.498318 7335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03d9ee425bb54b36ac818e1945008403: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:23.499179 7335 log.cc:824] T 00000000000000000000000000000000 P 03d9ee425bb54b36ac818e1945008403: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:23.501205 7335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03d9ee425bb54b36ac818e1945008403: No bootstrap required, opened a new log
Aug 12 03:07:23 dist-test-slave-6bbx krb5kdc[7205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 03:07:23.505211 7335 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03d9ee425bb54b36ac818e1945008403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d9ee425bb54b36ac818e1945008403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43491 } }
I20260812 03:07:23.505447 7335 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03d9ee425bb54b36ac818e1945008403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:23.505543 7335 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03d9ee425bb54b36ac818e1945008403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03d9ee425bb54b36ac818e1945008403, State: Initialized, Role: FOLLOWER
I20260812 03:07:23.506028 7335 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03d9ee425bb54b36ac818e1945008403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d9ee425bb54b36ac818e1945008403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43491 } }
I20260812 03:07:23.506191 7335 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03d9ee425bb54b36ac818e1945008403 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:07:23.506258 7335 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03d9ee425bb54b36ac818e1945008403 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:07:23.506356 7335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03d9ee425bb54b36ac818e1945008403 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:23.507319 7335 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03d9ee425bb54b36ac818e1945008403 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d9ee425bb54b36ac818e1945008403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43491 } }
I20260812 03:07:23.507659 7335 leader_election.cc:302] T 00000000000000000000000000000000 P 03d9ee425bb54b36ac818e1945008403 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03d9ee425bb54b36ac818e1945008403; no voters:
I20260812 03:07:23.508037 7335 leader_election.cc:288] T 00000000000000000000000000000000 P 03d9ee425bb54b36ac818e1945008403 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:07:23.508109 7343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03d9ee425bb54b36ac818e1945008403 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:23.508312 7343 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03d9ee425bb54b36ac818e1945008403 [term 1 LEADER]: Becoming Leader. State: Replica: 03d9ee425bb54b36ac818e1945008403, State: Running, Role: LEADER
I20260812 03:07:23.508644 7343 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03d9ee425bb54b36ac818e1945008403 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d9ee425bb54b36ac818e1945008403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43491 } }
I20260812 03:07:23.509272 7335 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03d9ee425bb54b36ac818e1945008403 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:23.510248 7344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03d9ee425bb54b36ac818e1945008403 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03d9ee425bb54b36ac818e1945008403" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d9ee425bb54b36ac818e1945008403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43491 } } }
I20260812 03:07:23.510247 7345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03d9ee425bb54b36ac818e1945008403 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03d9ee425bb54b36ac818e1945008403. Latest consensus state: current_term: 1 leader_uuid: "03d9ee425bb54b36ac818e1945008403" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d9ee425bb54b36ac818e1945008403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43491 } } }
I20260812 03:07:23.510391 7344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03d9ee425bb54b36ac818e1945008403 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:23.510391 7345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03d9ee425bb54b36ac818e1945008403 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:23.510524 7340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:23.496986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33495 (local address 127.0.0.1:43491)
0812 03:07:23.497351 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:23.497366 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:23.497399 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:07:23.497992 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:23.498008 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:23.498330 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:23.498610 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:23.499472 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:23.500610 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:23.501754 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:23.502217 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:23.502342 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:23.504986 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:23.504992 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:23.505007 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:23.505036 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:23.506945 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:23.507355 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:23.507361 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:23.507367 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:23.507503 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:23.507796 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:23.507799 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:23.507801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:23.508294 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:23.509126 (+ 832us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:23.509294 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 03:07:23.509577 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":92,"threads_started":1}
I20260812 03:07:23.510797 7358 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:23.513689 5506 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:07:23.513835 7358 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:23.518759 7358 catalog_manager.cc:1347] Generated new cluster ID: 94a7243b79db43339df7d896bbb8abb8
I20260812 03:07:23.518814 7358 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:23.524281 7340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:23.515826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45671 (local address 127.0.0.1:43491)
0812 03:07:23.515934 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:23.515938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:23.516323 (+ 385us) server_negotiation.cc:407] Connection header received
0812 03:07:23.516491 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:23.516494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:23.516537 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:23.516616 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:23.517370 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:23.518121 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:23.519247 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:23.519582 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:23.519716 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:23.521125 (+ 1409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:23.521129 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:23.521132 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:23.521155 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:23.522628 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:23.523060 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:23.523062 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:23.523064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:23.523134 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:23.523392 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:23.523395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:23.523397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:23.523561 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:23.523660 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:23.524028 (+ 368us) server_negotiation.cc:299] Negotiation successful
0812 03:07:23.524129 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 03:07:23.527521 7358 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:23.528663 7358 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:23.533660 7358 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03d9ee425bb54b36ac818e1945008403: Generated new TSK 0
I20260812 03:07:23.534312 7358 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:07:23.537796 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 7231
2026-08-12T03:07:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1494 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:07:23 dist-test-slave-6bbx krb5kdc[7384](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:07:23 dist-test-slave-6bbx krb5kdc[7384](info): set up 2 sockets
Aug 12 03:07:23 dist-test-slave-6bbx krb5kdc[7384](info): commencing operation
krb5kdc: starting...
W20260812 03:07:24.691378 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:07:24 dist-test-slave-6bbx krb5kdc[7384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:07:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:07:24.799247 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43245
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:24.917775 7402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:24.918004 7402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:24.918061 7402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:24.921967 7402 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:24.922017 7402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:24.922042 7402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:24.922075 7402 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:24.922114 7402 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:24.927095 7402 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:24.928020 7402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:24.928918 7402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:24.936751 7415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:24.936770 7418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:24.936807 7402 server_base.cc:1034] running on GCE node
W20260812 03:07:24.936761 7416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:24.937561 7402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:24.938527 7402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:24.939708 7402 hybrid_clock.cc:648] HybridClock initialized: now 1786504044939690 us; error 25 us; skew 500 ppm
Aug 12 03:07:24 dist-test-slave-6bbx krb5kdc[7384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504044, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:24.942191 7402 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:07:24.942953 7402 webserver.cc:460] Webserver started at http://127.0.0.1:42775/ using document root <none> and password file <none>
I20260812 03:07:24.943432 7402 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:24.943496 7402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:24.943732 7402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:24.945649 7402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "03a9a29b39b04c94a52602aee02d8c4e"
format_stamp: "Formatted at 2026-08-12 03:07:24 on dist-test-slave-6bbx"
server_key: "e8d92cf7b3395536c6e2adf89233381e"
server_key_iv: "68bdc47d12ffc488a42ef6fe493e70e6"
server_key_version: "encryptionkey@0"
I20260812 03:07:24.946167 7402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "03a9a29b39b04c94a52602aee02d8c4e"
format_stamp: "Formatted at 2026-08-12 03:07:24 on dist-test-slave-6bbx"
server_key: "e8d92cf7b3395536c6e2adf89233381e"
server_key_iv: "68bdc47d12ffc488a42ef6fe493e70e6"
server_key_version: "encryptionkey@0"
I20260812 03:07:24.949513 7402 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260812 03:07:24.951614 7428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:24.952687 7402 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 03:07:24.952806 7402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "03a9a29b39b04c94a52602aee02d8c4e"
format_stamp: "Formatted at 2026-08-12 03:07:24 on dist-test-slave-6bbx"
server_key: "e8d92cf7b3395536c6e2adf89233381e"
server_key_iv: "68bdc47d12ffc488a42ef6fe493e70e6"
server_key_version: "encryptionkey@0"
I20260812 03:07:24.952893 7402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:24.966591 7402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:24.968952 7402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:24.969102 7402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:24.975472 7402 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43245
I20260812 03:07:24.975517 7502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43245 every 8 connection(s)
I20260812 03:07:24.976307 7402 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:24.978817 7503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:24.983968 7503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03a9a29b39b04c94a52602aee02d8c4e: Bootstrap starting.
I20260812 03:07:24.984315 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 7402
I20260812 03:07:24.984462 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:07:24.984773 5506 external_mini_cluster.cc:1428] Setting key c2f306dd99137f1cecc887d2b8191234
I20260812 03:07:24.986491 7503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03a9a29b39b04c94a52602aee02d8c4e: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:24.987155 7503 log.cc:824] T 00000000000000000000000000000000 P 03a9a29b39b04c94a52602aee02d8c4e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:24.989109 7503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03a9a29b39b04c94a52602aee02d8c4e: No bootstrap required, opened a new log
I20260812 03:07:24.993073 7506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:24.985993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58613 (local address 127.0.0.1:43245)
0812 03:07:24.986408 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:24.986421 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:24.986454 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:07:24.987060 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:24.987076 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:24.987408 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:24.987779 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:24.988123 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:24.989056 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:24.990218 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:24.990654 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:24.990770 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:24.991506 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:24.991511 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:24.991523 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:24.991921 (+ 398us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:24.991973 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:24.991996 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:24.992068 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:24.992200 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 03:07:24.992373 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":124,"threads_started":1}
I20260812 03:07:24.993026 7503 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03a9a29b39b04c94a52602aee02d8c4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a9a29b39b04c94a52602aee02d8c4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43245 } }
I20260812 03:07:24.993221 7503 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03a9a29b39b04c94a52602aee02d8c4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:24.993279 7503 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03a9a29b39b04c94a52602aee02d8c4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a9a29b39b04c94a52602aee02d8c4e, State: Initialized, Role: FOLLOWER
I20260812 03:07:24.993677 7503 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03a9a29b39b04c94a52602aee02d8c4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a9a29b39b04c94a52602aee02d8c4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43245 } }
I20260812 03:07:24.993806 7503 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03a9a29b39b04c94a52602aee02d8c4e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:07:24.993896 7503 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03a9a29b39b04c94a52602aee02d8c4e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:07:24.993999 7503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03a9a29b39b04c94a52602aee02d8c4e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:24.994956 7503 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03a9a29b39b04c94a52602aee02d8c4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a9a29b39b04c94a52602aee02d8c4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43245 } }
I20260812 03:07:24.995292 7503 leader_election.cc:302] T 00000000000000000000000000000000 P 03a9a29b39b04c94a52602aee02d8c4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03a9a29b39b04c94a52602aee02d8c4e; no voters:
I20260812 03:07:24.995600 7503 leader_election.cc:288] T 00000000000000000000000000000000 P 03a9a29b39b04c94a52602aee02d8c4e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:07:24.995646 5506 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:07:24.995651 7509 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03a9a29b39b04c94a52602aee02d8c4e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:24.995862 7509 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03a9a29b39b04c94a52602aee02d8c4e [term 1 LEADER]: Becoming Leader. State: Replica: 03a9a29b39b04c94a52602aee02d8c4e, State: Running, Role: LEADER
I20260812 03:07:24.996129 7509 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03a9a29b39b04c94a52602aee02d8c4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a9a29b39b04c94a52602aee02d8c4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43245 } }
I20260812 03:07:24.996774 7503 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03a9a29b39b04c94a52602aee02d8c4e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:24.997666 7511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03a9a29b39b04c94a52602aee02d8c4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03a9a29b39b04c94a52602aee02d8c4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a9a29b39b04c94a52602aee02d8c4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43245 } } }
I20260812 03:07:24.997793 7511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03a9a29b39b04c94a52602aee02d8c4e [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:24.998067 7510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03a9a29b39b04c94a52602aee02d8c4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03a9a29b39b04c94a52602aee02d8c4e. Latest consensus state: current_term: 1 leader_uuid: "03a9a29b39b04c94a52602aee02d8c4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a9a29b39b04c94a52602aee02d8c4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43245 } } }
I20260812 03:07:24.998163 7510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03a9a29b39b04c94a52602aee02d8c4e [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:24.998178 7528 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:25.001219 7528 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:25.003925 7506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:24.997703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55839 (local address 127.0.0.1:43245)
0812 03:07:24.998296 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:24.998300 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:24.998312 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:07:24.998344 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:24.998346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:24.998384 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:24.998454 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:24.998715 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:25.001200 (+ 2485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:25.002260 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:25.002586 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:25.002693 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:25.003441 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:25.003444 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:25.003448 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:25.003460 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:25.003488 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:25.003501 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:25.003581 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:25.003743 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 03:07:25.003797 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521}
I20260812 03:07:25.006812 7528 catalog_manager.cc:1347] Generated new cluster ID: 0236f1a5d5b744b28cf168153d37a76b
I20260812 03:07:25.006875 7528 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:25.015453 7528 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:25.016598 7528 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:25.020988 7528 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03a9a29b39b04c94a52602aee02d8c4e: Generated new TSK 0
I20260812 03:07:25.021617 7528 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:07:25.030234 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 7402
2026-08-12T03:07:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1490 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:07:25 dist-test-slave-6bbx krb5kdc[7548](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:07:25 dist-test-slave-6bbx krb5kdc[7548](info): set up 2 sockets
Aug 12 03:07:25 dist-test-slave-6bbx krb5kdc[7548](info): commencing operation
krb5kdc: starting...
W20260812 03:07:26.199992 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:07:26 dist-test-slave-6bbx krb5kdc[7548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:07:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:07:26.310019 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41135
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:26.415891 7573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:26.416126 7573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:26.416180 7573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:26.420008 7573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:26.420058 7573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:26.420084 7573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:26.420106 7573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:26.420135 7573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:26.425086 7573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:26.425987 7573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:26.426899 7573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:26.434784 7583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:26.434834 7587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:26.434834 7584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:26.435087 7573 server_base.cc:1034] running on GCE node
I20260812 03:07:26.435642 7573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:26.436692 7573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:26.437839 7573 hybrid_clock.cc:648] HybridClock initialized: now 1786504046437825 us; error 25 us; skew 500 ppm
Aug 12 03:07:26 dist-test-slave-6bbx krb5kdc[7548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504046, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:26.445027 7573 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:07:26.445863 7573 webserver.cc:460] Webserver started at http://127.0.0.1:41355/ using document root <none> and password file <none>
I20260812 03:07:26.446363 7573 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:26.446420 7573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:26.446587 7573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:26.448553 7573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "a7c696b03db04250b63baa076f2005e6"
format_stamp: "Formatted at 2026-08-12 03:07:26 on dist-test-slave-6bbx"
server_key: "fa2f9838bf1577915376dbf03cf37578"
server_key_iv: "03bbba58794a32646a9bbf05ec2f4226"
server_key_version: "encryptionkey@0"
I20260812 03:07:26.449040 7573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "a7c696b03db04250b63baa076f2005e6"
format_stamp: "Formatted at 2026-08-12 03:07:26 on dist-test-slave-6bbx"
server_key: "fa2f9838bf1577915376dbf03cf37578"
server_key_iv: "03bbba58794a32646a9bbf05ec2f4226"
server_key_version: "encryptionkey@0"
I20260812 03:07:26.452746 7573 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:07:26.455088 7596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:26.456271 7573 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:07:26.456368 7573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "a7c696b03db04250b63baa076f2005e6"
format_stamp: "Formatted at 2026-08-12 03:07:26 on dist-test-slave-6bbx"
server_key: "fa2f9838bf1577915376dbf03cf37578"
server_key_iv: "03bbba58794a32646a9bbf05ec2f4226"
server_key_version: "encryptionkey@0"
I20260812 03:07:26.456451 7573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:26.460732 7573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:26.463291 7573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:26.463425 7573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:26.469941 7573 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41135
I20260812 03:07:26.469957 7678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41135 every 8 connection(s)
I20260812 03:07:26.470749 7573 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:26.473327 7679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:26.474750 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 7573
I20260812 03:07:26.474870 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:07:26.475095 5506 external_mini_cluster.cc:1428] Setting key d005b212953f5dbb795cf1da16d95f52
I20260812 03:07:26.478976 7679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7c696b03db04250b63baa076f2005e6: Bootstrap starting.
I20260812 03:07:26.479923 7682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:26.476382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41373 (local address 127.0.0.1:41135)
0812 03:07:26.476789 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:26.476803 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:26.476836 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:07:26.477467 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:26.477484 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:26.477744 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:26.478031 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:26.478759 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:26.478764 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:26.478775 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:26.478785 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:26.478836 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:26.478855 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:26.478978 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:26.479044 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 03:07:26.479211 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":127,"threads_started":1}
I20260812 03:07:26.481087 7679 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7c696b03db04250b63baa076f2005e6: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:26.481818 7679 log.cc:824] T 00000000000000000000000000000000 P a7c696b03db04250b63baa076f2005e6: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:26.482365 5506 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:07:26.483928 7679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7c696b03db04250b63baa076f2005e6: No bootstrap required, opened a new log
I20260812 03:07:26.485965 7682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:26.484555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41599 (local address 127.0.0.1:41135)
0812 03:07:26.484663 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:26.484666 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:26.484860 (+ 194us) server_negotiation.cc:407] Connection header received
0812 03:07:26.484953 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:26.484956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:26.484984 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:26.485045 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:26.485627 (+ 582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:26.485631 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:26.485633 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:26.485644 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:26.485669 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:26.485679 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:26.485746 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:26.485828 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 03:07:26.485861 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 03:07:26.487504 7679 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7c696b03db04250b63baa076f2005e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7c696b03db04250b63baa076f2005e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41135 } }
I20260812 03:07:26.487735 7679 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7c696b03db04250b63baa076f2005e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:26.487782 7679 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7c696b03db04250b63baa076f2005e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7c696b03db04250b63baa076f2005e6, State: Initialized, Role: FOLLOWER
I20260812 03:07:26.488142 7679 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7c696b03db04250b63baa076f2005e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7c696b03db04250b63baa076f2005e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41135 } }
I20260812 03:07:26.488262 7679 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7c696b03db04250b63baa076f2005e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:07:26.488307 7679 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7c696b03db04250b63baa076f2005e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:07:26.488373 7679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7c696b03db04250b63baa076f2005e6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:26.489123 7679 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7c696b03db04250b63baa076f2005e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7c696b03db04250b63baa076f2005e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41135 } }
I20260812 03:07:26.489404 7679 leader_election.cc:302] T 00000000000000000000000000000000 P a7c696b03db04250b63baa076f2005e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7c696b03db04250b63baa076f2005e6; no voters:
I20260812 03:07:26.489651 7679 leader_election.cc:288] T 00000000000000000000000000000000 P a7c696b03db04250b63baa076f2005e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:07:26.489751 7692 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7c696b03db04250b63baa076f2005e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:26.489940 7692 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7c696b03db04250b63baa076f2005e6 [term 1 LEADER]: Becoming Leader. State: Replica: a7c696b03db04250b63baa076f2005e6, State: Running, Role: LEADER
I20260812 03:07:26.490438 7692 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7c696b03db04250b63baa076f2005e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7c696b03db04250b63baa076f2005e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41135 } }
I20260812 03:07:26.490800 7679 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7c696b03db04250b63baa076f2005e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:26.491938 7695 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7c696b03db04250b63baa076f2005e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7c696b03db04250b63baa076f2005e6. Latest consensus state: current_term: 1 leader_uuid: "a7c696b03db04250b63baa076f2005e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7c696b03db04250b63baa076f2005e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41135 } } }
I20260812 03:07:26.492062 7695 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7c696b03db04250b63baa076f2005e6 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:26.492350 7694 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7c696b03db04250b63baa076f2005e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7c696b03db04250b63baa076f2005e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7c696b03db04250b63baa076f2005e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41135 } } }
I20260812 03:07:26.492442 7694 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7c696b03db04250b63baa076f2005e6 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:26.492568 7706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:26.495041 7706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:26.500180 7706 catalog_manager.cc:1347] Generated new cluster ID: dd7606e5403f4b78bf4ad3b4eba789f5
I20260812 03:07:26.500243 7706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:26.514634 7706 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:26.514894 7706 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:26.521406 7706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7c696b03db04250b63baa076f2005e6: Generated new TSK 0
I20260812 03:07:26.521947 7706 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:07:26.529366 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 7573
2026-08-12T03:07:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1516 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:07:26 dist-test-slave-6bbx krb5kdc[7720](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:07:26 dist-test-slave-6bbx krb5kdc[7720](info): set up 2 sockets
Aug 12 03:07:26 dist-test-slave-6bbx krb5kdc[7720](info): commencing operation
krb5kdc: starting...
W20260812 03:07:27.698880 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:07:27 dist-test-slave-6bbx krb5kdc[7720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504047, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:07:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:07:27.808118 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41541
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:27.916330 7737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:27.916551 7737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:27.916606 7737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:27.920487 7737 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:27.920540 7737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:27.920565 7737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:27.920596 7737 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:27.920625 7737 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:27.925911 7737 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:27.926802 7737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:27.927732 7737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:27.935443 7744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:27.935460 7747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:27.935461 7743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:27.935621 7737 server_base.cc:1034] running on GCE node
I20260812 03:07:27.936172 7737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:27.937160 7737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:27.938328 7737 hybrid_clock.cc:648] HybridClock initialized: now 1786504047938300 us; error 42 us; skew 500 ppm
Aug 12 03:07:27 dist-test-slave-6bbx krb5kdc[7720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504047, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:27.941004 7737 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:07:27.941807 7737 webserver.cc:460] Webserver started at http://127.0.0.1:46473/ using document root <none> and password file <none>
I20260812 03:07:27.942306 7737 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:27.942363 7737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:27.942526 7737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:27.944468 7737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "937f0ca0c2af40cf9eb0905a8264eb6e"
format_stamp: "Formatted at 2026-08-12 03:07:27 on dist-test-slave-6bbx"
server_key: "c82fd607ca03f700d83bcbd1fb6e6ce0"
server_key_iv: "514a411ab5681cb20f0eff0b0a83d59f"
server_key_version: "encryptionkey@0"
I20260812 03:07:27.944957 7737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "937f0ca0c2af40cf9eb0905a8264eb6e"
format_stamp: "Formatted at 2026-08-12 03:07:27 on dist-test-slave-6bbx"
server_key: "c82fd607ca03f700d83bcbd1fb6e6ce0"
server_key_iv: "514a411ab5681cb20f0eff0b0a83d59f"
server_key_version: "encryptionkey@0"
I20260812 03:07:27.948385 7737 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:07:27.950560 7756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:27.951650 7737 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:07:27.951800 7737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "937f0ca0c2af40cf9eb0905a8264eb6e"
format_stamp: "Formatted at 2026-08-12 03:07:27 on dist-test-slave-6bbx"
server_key: "c82fd607ca03f700d83bcbd1fb6e6ce0"
server_key_iv: "514a411ab5681cb20f0eff0b0a83d59f"
server_key_version: "encryptionkey@0"
I20260812 03:07:27.951889 7737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:27.969641 7737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:27.972891 7737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:27.973061 7737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:27.982550 7737 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41541
I20260812 03:07:27.982553 7843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41541 every 8 connection(s)
I20260812 03:07:27.983465 7737 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:27.986099 7844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:27.991271 7844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 937f0ca0c2af40cf9eb0905a8264eb6e: Bootstrap starting.
I20260812 03:07:27.993360 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 7737
I20260812 03:07:27.993470 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:07:27.993608 7844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 937f0ca0c2af40cf9eb0905a8264eb6e: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:27.993722 5506 external_mini_cluster.cc:1428] Setting key e205fc2de029dd2af211e1fbd14446ca
I20260812 03:07:27.994406 7844 log.cc:824] T 00000000000000000000000000000000 P 937f0ca0c2af40cf9eb0905a8264eb6e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:27.996599 7844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 937f0ca0c2af40cf9eb0905a8264eb6e: No bootstrap required, opened a new log
I20260812 03:07:28.000692 7844 raft_consensus.cc:348] T 00000000000000000000000000000000 P 937f0ca0c2af40cf9eb0905a8264eb6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "937f0ca0c2af40cf9eb0905a8264eb6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41541 } }
I20260812 03:07:28.000941 7844 raft_consensus.cc:374] T 00000000000000000000000000000000 P 937f0ca0c2af40cf9eb0905a8264eb6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:28.001012 7844 raft_consensus.cc:729] T 00000000000000000000000000000000 P 937f0ca0c2af40cf9eb0905a8264eb6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 937f0ca0c2af40cf9eb0905a8264eb6e, State: Initialized, Role: FOLLOWER
I20260812 03:07:28.001447 7844 consensus_queue.cc:260] T 00000000000000000000000000000000 P 937f0ca0c2af40cf9eb0905a8264eb6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "937f0ca0c2af40cf9eb0905a8264eb6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41541 } }
I20260812 03:07:28.001576 7844 raft_consensus.cc:388] T 00000000000000000000000000000000 P 937f0ca0c2af40cf9eb0905a8264eb6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:07:28.001643 7844 raft_consensus.cc:482] T 00000000000000000000000000000000 P 937f0ca0c2af40cf9eb0905a8264eb6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:07:28.001740 7844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 937f0ca0c2af40cf9eb0905a8264eb6e [term 0 FOLLOWER]: Advancing to term 1
Aug 12 03:07:27 dist-test-slave-6bbx krb5kdc[7720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504047, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 03:07:28.002580 7844 raft_consensus.cc:504] T 00000000000000000000000000000000 P 937f0ca0c2af40cf9eb0905a8264eb6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "937f0ca0c2af40cf9eb0905a8264eb6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41541 } }
I20260812 03:07:28.002908 7844 leader_election.cc:302] T 00000000000000000000000000000000 P 937f0ca0c2af40cf9eb0905a8264eb6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 937f0ca0c2af40cf9eb0905a8264eb6e; no voters:
I20260812 03:07:28.003249 7844 leader_election.cc:288] T 00000000000000000000000000000000 P 937f0ca0c2af40cf9eb0905a8264eb6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:07:28.003315 7853 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 937f0ca0c2af40cf9eb0905a8264eb6e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:28.003544 7853 raft_consensus.cc:686] T 00000000000000000000000000000000 P 937f0ca0c2af40cf9eb0905a8264eb6e [term 1 LEADER]: Becoming Leader. State: Replica: 937f0ca0c2af40cf9eb0905a8264eb6e, State: Running, Role: LEADER
I20260812 03:07:28.003903 7853 consensus_queue.cc:237] T 00000000000000000000000000000000 P 937f0ca0c2af40cf9eb0905a8264eb6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "937f0ca0c2af40cf9eb0905a8264eb6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41541 } }
I20260812 03:07:28.004511 7844 sys_catalog.cc:564] T 00000000000000000000000000000000 P 937f0ca0c2af40cf9eb0905a8264eb6e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:28.005427 7856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 937f0ca0c2af40cf9eb0905a8264eb6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 937f0ca0c2af40cf9eb0905a8264eb6e. Latest consensus state: current_term: 1 leader_uuid: "937f0ca0c2af40cf9eb0905a8264eb6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "937f0ca0c2af40cf9eb0905a8264eb6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41541 } } }
I20260812 03:07:28.005555 7856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 937f0ca0c2af40cf9eb0905a8264eb6e [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:28.005887 7855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 937f0ca0c2af40cf9eb0905a8264eb6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "937f0ca0c2af40cf9eb0905a8264eb6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "937f0ca0c2af40cf9eb0905a8264eb6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41541 } } }
I20260812 03:07:28.005990 7855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 937f0ca0c2af40cf9eb0905a8264eb6e [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:28.008759 7851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:27.994807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:40497 (local address 127.0.0.1:41541)
0812 03:07:27.995222 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:27.995242 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:27.995284 (+ 42us) server_negotiation.cc:407] Connection header received
0812 03:07:27.996118 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:27.996139 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:27.996531 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:27.996914 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:27.997545 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:27.999032 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:27.999846 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:28.000525 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:28.000676 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:28.002974 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:28.003006 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:28.003023 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:28.003054 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:28.005387 (+ 2333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:28.006031 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:28.006037 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:28.006042 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:28.006192 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:28.006603 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:28.006606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:28.006608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:28.007090 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:28.007299 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:28.007607 (+ 308us) server_negotiation.cc:299] Negotiation successful
0812 03:07:28.007859 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":110,"threads_started":1}
W20260812 03:07:28.009450 7879 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 937f0ca0c2af40cf9eb0905a8264eb6e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:07:28.009522 7879 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:07:28.009588 7869 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:28.010435 7869 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:28.011744 5506 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:07:28.016299 7869 catalog_manager.cc:1347] Generated new cluster ID: 531945f8e8684667b40ec4881110441c
I20260812 03:07:28.016364 7869 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:28.021014 7851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:28.013524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:54715 (local address 127.0.0.1:41541)
0812 03:07:28.013623 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:28.013627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:28.013946 (+ 319us) server_negotiation.cc:407] Connection header received
0812 03:07:28.014048 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:28.014051 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:28.014090 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:28.014147 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:28.014765 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:28.015528 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:28.016582 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:28.016840 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:28.016926 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:28.018128 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:28.018131 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:28.018134 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:28.018155 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:28.019378 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:28.019839 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:28.019843 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:28.019845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:28.019914 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:28.020214 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:28.020217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:28.020218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:28.020370 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:28.020492 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:28.020756 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 03:07:28.020838 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 03:07:28.050375 7869 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:28.051194 7869 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:28.057621 7869 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 937f0ca0c2af40cf9eb0905a8264eb6e: Generated new TSK 0
I20260812 03:07:28.058182 7869 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:07:28.062170 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 7737
W20260812 03:07:28.077536 7718 connection.cc:535] client connection to 127.0.0.1:41541 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41541): Connection reset by peer (error 104)
2026-08-12T03:07:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1516 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:07:28 dist-test-slave-6bbx krb5kdc[7893](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:07:28 dist-test-slave-6bbx krb5kdc[7893](info): set up 2 sockets
Aug 12 03:07:28 dist-test-slave-6bbx krb5kdc[7893](info): commencing operation
krb5kdc: starting...
W20260812 03:07:29.220054 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:07:29 dist-test-slave-6bbx krb5kdc[7893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:07:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:07:29.329473 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42743
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:29.436149 7911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:29.436389 7911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:29.436434 7911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:29.440351 7911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:29.440398 7911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:29.440418 7911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:29.440446 7911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:29.440474 7911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:29.445526 7911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:29.446416 7911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:29.447366 7911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:29.455068 7917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:29.455090 7920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:29.455068 7925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:29.455294 7911 server_base.cc:1034] running on GCE node
I20260812 03:07:29.455946 7911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:29.456938 7911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:29.458091 7911 hybrid_clock.cc:648] HybridClock initialized: now 1786504049458074 us; error 27 us; skew 500 ppm
Aug 12 03:07:29 dist-test-slave-6bbx krb5kdc[7893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504049, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:29.460678 7911 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:07:29.461452 7911 webserver.cc:460] Webserver started at http://127.0.0.1:46243/ using document root <none> and password file <none>
I20260812 03:07:29.461936 7911 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:29.462002 7911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:29.462194 7911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:29.464133 7911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "6bb311d03ea042ca84c5444c71de33a9"
format_stamp: "Formatted at 2026-08-12 03:07:29 on dist-test-slave-6bbx"
server_key: "000e6cb7fb46e5127f6dbe59dc30fd16"
server_key_iv: "c16f61203e030427503a0c498e45b596"
server_key_version: "encryptionkey@0"
I20260812 03:07:29.464655 7911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "6bb311d03ea042ca84c5444c71de33a9"
format_stamp: "Formatted at 2026-08-12 03:07:29 on dist-test-slave-6bbx"
server_key: "000e6cb7fb46e5127f6dbe59dc30fd16"
server_key_iv: "c16f61203e030427503a0c498e45b596"
server_key_version: "encryptionkey@0"
I20260812 03:07:29.468060 7911 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:07:29.470201 7934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:29.471261 7911 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:07:29.471369 7911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "6bb311d03ea042ca84c5444c71de33a9"
format_stamp: "Formatted at 2026-08-12 03:07:29 on dist-test-slave-6bbx"
server_key: "000e6cb7fb46e5127f6dbe59dc30fd16"
server_key_iv: "c16f61203e030427503a0c498e45b596"
server_key_version: "encryptionkey@0"
I20260812 03:07:29.471457 7911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:29.488480 7911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:29.490818 7911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:29.490974 7911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:29.497457 7911 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42743
I20260812 03:07:29.497470 8014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42743 every 8 connection(s)
I20260812 03:07:29.498283 7911 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:29.500806 8015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:29.504511 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 7911
I20260812 03:07:29.504628 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:07:29.504871 5506 external_mini_cluster.cc:1428] Setting key 2a24469dd16ccf3855479473f61ad73c
I20260812 03:07:29.506050 8015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb311d03ea042ca84c5444c71de33a9: Bootstrap starting.
I20260812 03:07:29.508222 8015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6bb311d03ea042ca84c5444c71de33a9: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:29.508905 8015 log.cc:824] T 00000000000000000000000000000000 P 6bb311d03ea042ca84c5444c71de33a9: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:29.510555 8015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb311d03ea042ca84c5444c71de33a9: No bootstrap required, opened a new log
I20260812 03:07:29.512660 8021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:29.506018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:46797 (local address 127.0.0.1:42743)
0812 03:07:29.506391 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:29.506406 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:29.506438 (+ 32us) server_negotiation.cc:407] Connection header received
0812 03:07:29.507036 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:29.507053 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:29.507364 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:29.507615 (+ 251us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:29.507959 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:29.508920 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:29.509797 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:29.510240 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:29.510392 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:29.511183 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:29.511189 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:29.511200 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:29.511535 (+ 335us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:29.511583 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:29.511605 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:29.511751 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:29.511918 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 03:07:29.512036 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":120,"threads_started":1}
I20260812 03:07:29.514211 8015 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bb311d03ea042ca84c5444c71de33a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb311d03ea042ca84c5444c71de33a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42743 } }
I20260812 03:07:29.514451 8015 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6bb311d03ea042ca84c5444c71de33a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:29.514556 8015 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bb311d03ea042ca84c5444c71de33a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bb311d03ea042ca84c5444c71de33a9, State: Initialized, Role: FOLLOWER
I20260812 03:07:29.514978 8015 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bb311d03ea042ca84c5444c71de33a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb311d03ea042ca84c5444c71de33a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42743 } }
I20260812 03:07:29.515110 8015 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6bb311d03ea042ca84c5444c71de33a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:07:29.515213 8015 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb311d03ea042ca84c5444c71de33a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:07:29.515336 5506 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:07:29.515334 8015 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bb311d03ea042ca84c5444c71de33a9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:29.516304 8015 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb311d03ea042ca84c5444c71de33a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb311d03ea042ca84c5444c71de33a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42743 } }
I20260812 03:07:29.516666 8015 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb311d03ea042ca84c5444c71de33a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6bb311d03ea042ca84c5444c71de33a9; no voters:
I20260812 03:07:29.517021 8015 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb311d03ea042ca84c5444c71de33a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:07:29.517094 8031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bb311d03ea042ca84c5444c71de33a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:29.517321 8031 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bb311d03ea042ca84c5444c71de33a9 [term 1 LEADER]: Becoming Leader. State: Replica: 6bb311d03ea042ca84c5444c71de33a9, State: Running, Role: LEADER
I20260812 03:07:29.517656 8031 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bb311d03ea042ca84c5444c71de33a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb311d03ea042ca84c5444c71de33a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42743 } }
I20260812 03:07:29.518149 8015 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bb311d03ea042ca84c5444c71de33a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:29.519322 8034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb311d03ea042ca84c5444c71de33a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bb311d03ea042ca84c5444c71de33a9. Latest consensus state: current_term: 1 leader_uuid: "6bb311d03ea042ca84c5444c71de33a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb311d03ea042ca84c5444c71de33a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42743 } } }
I20260812 03:07:29.519328 8032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb311d03ea042ca84c5444c71de33a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6bb311d03ea042ca84c5444c71de33a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb311d03ea042ca84c5444c71de33a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42743 } } }
I20260812 03:07:29.519522 8034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb311d03ea042ca84c5444c71de33a9 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:29.519526 8032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb311d03ea042ca84c5444c71de33a9 [sys.catalog]: This master's current role is: LEADER
W20260812 03:07:29.522853 8060 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6bb311d03ea042ca84c5444c71de33a9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:07:29.522967 8060 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:07:29.523061 8021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:29.517745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:42925 (local address 127.0.0.1:42743)
0812 03:07:29.517853 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:29.517857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:29.518875 (+ 1018us) server_negotiation.cc:407] Connection header received
0812 03:07:29.519048 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:29.519051 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:29.519093 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:29.519141 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:29.519564 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:29.520269 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:29.521333 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:29.521641 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:29.521756 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:29.522556 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:29.522560 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:29.522563 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:29.522575 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:29.522603 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:29.522615 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:29.522725 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:29.522879 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 03:07:29.522911 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:07:29.523080 8052 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:29.524001 8052 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:29.529177 8052 catalog_manager.cc:1347] Generated new cluster ID: b1aef1ff1b6d4632860253201a0c4787
I20260812 03:07:29.529228 8052 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:29.539422 8052 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:29.540469 8052 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:29.546425 8052 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6bb311d03ea042ca84c5444c71de33a9: Generated new TSK 0
I20260812 03:07:29.547079 8052 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:07:29.557885 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 7911
W20260812 03:07:29.575750 7891 connection.cc:535] client connection to 127.0.0.1:42743 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42743): Connection reset by peer (error 104)
2026-08-12T03:07:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1498 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:07:29 dist-test-slave-6bbx krb5kdc[8068](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:07:29 dist-test-slave-6bbx krb5kdc[8068](info): set up 2 sockets
Aug 12 03:07:29 dist-test-slave-6bbx krb5kdc[8068](info): commencing operation
krb5kdc: starting...
W20260812 03:07:30.711778 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:07:30 dist-test-slave-6bbx krb5kdc[8068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:07:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:07:30.822738 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44379
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44379
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:30.930155 8088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:30.930387 8088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:30.930441 8088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:30.934613 8088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:30.934675 8088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:30.934701 8088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:30.934723 8088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:30.934751 8088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:30.939932 8088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:30.940857 8088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:30.941778 8088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:30.949741 8093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:30.949764 8098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:30.949738 8094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:30.949816 8088 server_base.cc:1034] running on GCE node
I20260812 03:07:30.950500 8088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:30.951455 8088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:30.952607 8088 hybrid_clock.cc:648] HybridClock initialized: now 1786504050952585 us; error 32 us; skew 500 ppm
Aug 12 03:07:30 dist-test-slave-6bbx krb5kdc[8068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504050, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:30.955147 8088 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:07:30.956015 8088 webserver.cc:460] Webserver started at http://127.0.0.1:36545/ using document root <none> and password file <none>
I20260812 03:07:30.956507 8088 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:30.956574 8088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:30.956781 8088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:30.958676 8088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "a2cff538f49446979f461af174549aa1"
format_stamp: "Formatted at 2026-08-12 03:07:30 on dist-test-slave-6bbx"
server_key: "198b45237aa92eb2f60d26d48b327da4"
server_key_iv: "099fef99bcd38bc2ca9ec5b7cbecb8f7"
server_key_version: "encryptionkey@0"
I20260812 03:07:30.959199 8088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "a2cff538f49446979f461af174549aa1"
format_stamp: "Formatted at 2026-08-12 03:07:30 on dist-test-slave-6bbx"
server_key: "198b45237aa92eb2f60d26d48b327da4"
server_key_iv: "099fef99bcd38bc2ca9ec5b7cbecb8f7"
server_key_version: "encryptionkey@0"
I20260812 03:07:30.962603 8088 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 03:07:30.964823 8107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:30.965899 8088 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:07:30.966020 8088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "a2cff538f49446979f461af174549aa1"
format_stamp: "Formatted at 2026-08-12 03:07:30 on dist-test-slave-6bbx"
server_key: "198b45237aa92eb2f60d26d48b327da4"
server_key_iv: "099fef99bcd38bc2ca9ec5b7cbecb8f7"
server_key_version: "encryptionkey@0"
I20260812 03:07:30.966109 8088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:30.970485 8088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:30.973316 8088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:30.973471 8088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:30.979992 8088 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44379
I20260812 03:07:30.980010 8173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44379 every 8 connection(s)
I20260812 03:07:30.980803 8088 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:30.983323 8178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:30.987543 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 8088
I20260812 03:07:30.987682 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:07:30.987954 5506 external_mini_cluster.cc:1428] Setting key 33a16f0950830498dc270cfea118578e
I20260812 03:07:30.988437 8178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2cff538f49446979f461af174549aa1: Bootstrap starting.
I20260812 03:07:30.990653 8178 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2cff538f49446979f461af174549aa1: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:30.991503 8178 log.cc:824] T 00000000000000000000000000000000 P a2cff538f49446979f461af174549aa1: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:30.992765 8183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:30.989132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:34999 (local address 127.0.0.1:44379)
0812 03:07:30.989520 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:30.989534 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:30.989569 (+ 35us) server_negotiation.cc:407] Connection header received
0812 03:07:30.990219 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:30.990237 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:30.990504 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:30.990793 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:30.991530 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:30.991534 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:30.991546 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:30.991555 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:30.991621 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:30.991648 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:30.991763 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:30.991887 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 03:07:30.992044 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":100,"threads_started":1}
I20260812 03:07:30.993407 8178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2cff538f49446979f461af174549aa1: No bootstrap required, opened a new log
I20260812 03:07:30.995406 5506 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:07:30.997668 8178 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2cff538f49446979f461af174549aa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2cff538f49446979f461af174549aa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44379 } }
I20260812 03:07:30.997908 8178 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2cff538f49446979f461af174549aa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:30.997996 8178 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2cff538f49446979f461af174549aa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2cff538f49446979f461af174549aa1, State: Initialized, Role: FOLLOWER
I20260812 03:07:30.998449 8178 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2cff538f49446979f461af174549aa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2cff538f49446979f461af174549aa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44379 } }
I20260812 03:07:30.998618 8178 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2cff538f49446979f461af174549aa1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:07:30.998689 8178 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2cff538f49446979f461af174549aa1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:07:30.998798 8178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2cff538f49446979f461af174549aa1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:30.999105 8183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:30.997580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:50659 (local address 127.0.0.1:44379)
0812 03:07:30.997690 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:30.997694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:30.997897 (+ 203us) server_negotiation.cc:407] Connection header received
0812 03:07:30.997999 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:30.998002 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:30.998034 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:30.998114 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:30.998704 (+ 590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:30.998707 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:30.998710 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:30.998723 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:30.998750 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:30.998764 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:30.998845 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:30.998965 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 03:07:30.999005 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:07:30.999802 8178 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2cff538f49446979f461af174549aa1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2cff538f49446979f461af174549aa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44379 } }
I20260812 03:07:31.000123 8178 leader_election.cc:302] T 00000000000000000000000000000000 P a2cff538f49446979f461af174549aa1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2cff538f49446979f461af174549aa1; no voters:
I20260812 03:07:31.000383 8178 leader_election.cc:288] T 00000000000000000000000000000000 P a2cff538f49446979f461af174549aa1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:07:31.000478 8193 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2cff538f49446979f461af174549aa1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:31.000676 8193 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2cff538f49446979f461af174549aa1 [term 1 LEADER]: Becoming Leader. State: Replica: a2cff538f49446979f461af174549aa1, State: Running, Role: LEADER
I20260812 03:07:31.001019 8193 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2cff538f49446979f461af174549aa1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2cff538f49446979f461af174549aa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44379 } }
I20260812 03:07:31.001464 8178 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2cff538f49446979f461af174549aa1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:31.002472 8200 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2cff538f49446979f461af174549aa1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2cff538f49446979f461af174549aa1. Latest consensus state: current_term: 1 leader_uuid: "a2cff538f49446979f461af174549aa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2cff538f49446979f461af174549aa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44379 } } }
I20260812 03:07:31.002591 8200 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2cff538f49446979f461af174549aa1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:31.002974 8209 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:31.006029 8209 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:31.002940 8194 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2cff538f49446979f461af174549aa1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2cff538f49446979f461af174549aa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2cff538f49446979f461af174549aa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44379 } } }
I20260812 03:07:31.006639 8194 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2cff538f49446979f461af174549aa1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:31.011193 8209 catalog_manager.cc:1347] Generated new cluster ID: bf8589ce7c974aee8d415fceb61509c5
I20260812 03:07:31.011256 8209 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:31.029300 8209 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:31.029589 8209 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:31.035310 8209 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2cff538f49446979f461af174549aa1: Generated new TSK 0
I20260812 03:07:31.036002 8209 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:07:31.044098 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 8088
2026-08-12T03:07:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1478 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:07:31 dist-test-slave-6bbx krb5kdc[8226](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:07:31 dist-test-slave-6bbx krb5kdc[8226](info): set up 2 sockets
Aug 12 03:07:31 dist-test-slave-6bbx krb5kdc[8226](info): commencing operation
krb5kdc: starting...
W20260812 03:07:32.200985 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:07:32 dist-test-slave-6bbx krb5kdc[8226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:07:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:07:32.311503 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46297
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:32.418545 8246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:32.418777 8246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:32.418834 8246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:32.422717 8246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:32.422771 8246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:32.422797 8246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:32.422820 8246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:32.422849 8246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:32.427837 8246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46297
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:32.428723 8246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:32.429628 8246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:32.437501 8251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:32.437498 8254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:32.437501 8260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:32.437546 8246 server_base.cc:1034] running on GCE node
I20260812 03:07:32.438215 8246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:32.439248 8246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:32.440438 8246 hybrid_clock.cc:648] HybridClock initialized: now 1786504052440413 us; error 34 us; skew 500 ppm
Aug 12 03:07:32 dist-test-slave-6bbx krb5kdc[8226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504052, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:32.446123 8246 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:07:32.446965 8246 webserver.cc:460] Webserver started at http://127.0.0.1:40055/ using document root <none> and password file <none>
I20260812 03:07:32.447463 8246 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:32.447527 8246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:32.447796 8246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:32.449715 8246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "5fcb17f920844f0481e033973aba7bf7"
format_stamp: "Formatted at 2026-08-12 03:07:32 on dist-test-slave-6bbx"
server_key: "641727180e5bf0aad88f591f179e2001"
server_key_iv: "8207be62af864db444ad9e846da726cd"
server_key_version: "encryptionkey@0"
I20260812 03:07:32.450243 8246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "5fcb17f920844f0481e033973aba7bf7"
format_stamp: "Formatted at 2026-08-12 03:07:32 on dist-test-slave-6bbx"
server_key: "641727180e5bf0aad88f591f179e2001"
server_key_iv: "8207be62af864db444ad9e846da726cd"
server_key_version: "encryptionkey@0"
I20260812 03:07:32.453738 8246 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 03:07:32.455996 8270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:32.457729 8246 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:07:32.457854 8246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "5fcb17f920844f0481e033973aba7bf7"
format_stamp: "Formatted at 2026-08-12 03:07:32 on dist-test-slave-6bbx"
server_key: "641727180e5bf0aad88f591f179e2001"
server_key_iv: "8207be62af864db444ad9e846da726cd"
server_key_version: "encryptionkey@0"
I20260812 03:07:32.457949 8246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:32.480262 8246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:32.482707 8246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:32.482971 8246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:32.489686 8246 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46297
I20260812 03:07:32.489702 8347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46297 every 8 connection(s)
I20260812 03:07:32.490582 8246 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:32.493193 8348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:32.496521 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 8246
I20260812 03:07:32.496630 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:07:32.496863 5506 external_mini_cluster.cc:1428] Setting key 4e3d0d322471da80f2a573353db40a2b
I20260812 03:07:32.498824 8348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fcb17f920844f0481e033973aba7bf7: Bootstrap starting.
I20260812 03:07:32.501495 8348 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fcb17f920844f0481e033973aba7bf7: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:32.502362 8348 log.cc:824] T 00000000000000000000000000000000 P 5fcb17f920844f0481e033973aba7bf7: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:32.504453 8348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fcb17f920844f0481e033973aba7bf7: No bootstrap required, opened a new log
Aug 12 03:07:32 dist-test-slave-6bbx krb5kdc[8226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 03:07:32.508289 8348 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fcb17f920844f0481e033973aba7bf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fcb17f920844f0481e033973aba7bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46297 } }
I20260812 03:07:32.508546 8348 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5fcb17f920844f0481e033973aba7bf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:32.508630 8348 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fcb17f920844f0481e033973aba7bf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fcb17f920844f0481e033973aba7bf7, State: Initialized, Role: FOLLOWER
I20260812 03:07:32.509073 8348 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fcb17f920844f0481e033973aba7bf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fcb17f920844f0481e033973aba7bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46297 } }
I20260812 03:07:32.509235 8348 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5fcb17f920844f0481e033973aba7bf7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:07:32.509305 8348 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5fcb17f920844f0481e033973aba7bf7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:07:32.509402 8348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fcb17f920844f0481e033973aba7bf7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:32.510181 8348 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5fcb17f920844f0481e033973aba7bf7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fcb17f920844f0481e033973aba7bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46297 } }
I20260812 03:07:32.510525 8348 leader_election.cc:302] T 00000000000000000000000000000000 P 5fcb17f920844f0481e033973aba7bf7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5fcb17f920844f0481e033973aba7bf7; no voters:
I20260812 03:07:32.510842 8348 leader_election.cc:288] T 00000000000000000000000000000000 P 5fcb17f920844f0481e033973aba7bf7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:07:32.510934 8357 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5fcb17f920844f0481e033973aba7bf7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:32.511103 8357 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5fcb17f920844f0481e033973aba7bf7 [term 1 LEADER]: Becoming Leader. State: Replica: 5fcb17f920844f0481e033973aba7bf7, State: Running, Role: LEADER
I20260812 03:07:32.511381 8353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:32.498094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:34501 (local address 127.0.0.1:46297)
0812 03:07:32.498546 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:32.498564 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:32.498606 (+ 42us) server_negotiation.cc:407] Connection header received
0812 03:07:32.499444 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:32.499468 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:32.499912 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:32.500286 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:32.500925 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:32.502395 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:32.503417 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:32.504113 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:32.504277 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:32.506508 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:32.506514 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:32.506531 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:32.506562 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:32.508418 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:32.508954 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:32.508960 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:32.508965 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:32.509118 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:32.509456 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:32.509459 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:32.509460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:32.509839 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:32.510039 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:32.510370 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 03:07:32.510563 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":135,"threads_started":1}
I20260812 03:07:32.511438 8357 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5fcb17f920844f0481e033973aba7bf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fcb17f920844f0481e033973aba7bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46297 } }
I20260812 03:07:32.512063 8348 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fcb17f920844f0481e033973aba7bf7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:32.512611 8357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fcb17f920844f0481e033973aba7bf7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fcb17f920844f0481e033973aba7bf7. Latest consensus state: current_term: 1 leader_uuid: "5fcb17f920844f0481e033973aba7bf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fcb17f920844f0481e033973aba7bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46297 } } }
I20260812 03:07:32.512727 8357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fcb17f920844f0481e033973aba7bf7 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:32.512959 8358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fcb17f920844f0481e033973aba7bf7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5fcb17f920844f0481e033973aba7bf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fcb17f920844f0481e033973aba7bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46297 } } }
I20260812 03:07:32.513051 8358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fcb17f920844f0481e033973aba7bf7 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:32.513463 8368 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:32.514560 5506 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:07:32.516489 8368 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:32.521718 8368 catalog_manager.cc:1347] Generated new cluster ID: 4836db4bf8c34386b0cbc75143593ddc
I20260812 03:07:32.521781 8368 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:32.537786 8368 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:32.538798 8368 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:32.539907 8353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:32.520036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:38077 (local address 127.0.0.1:46297)
0812 03:07:32.520132 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:32.520136 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:32.531901 (+ 11765us) server_negotiation.cc:407] Connection header received
0812 03:07:32.531994 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:32.531997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:32.532041 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:32.532109 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:32.532856 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:32.533592 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:32.534697 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:32.535044 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:32.535127 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:32.536648 (+ 1521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:32.536652 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:32.536655 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:32.536680 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:32.538115 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:32.538624 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:32.538627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:32.538629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:32.538714 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:32.539044 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:32.539048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:32.539049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:32.539254 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:32.539401 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:32.539659 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 03:07:32.539767 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 03:07:32.542845 8368 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5fcb17f920844f0481e033973aba7bf7: Generated new TSK 0
I20260812 03:07:32.543371 8368 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:07:32.547698 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 8246
W20260812 03:07:32.566751 8223 connection.cc:535] client connection to 127.0.0.1:46297 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46297): Connection reset by peer (error 104)
2026-08-12T03:07:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1512 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:07:32 dist-test-slave-6bbx krb5kdc[8403](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:07:32 dist-test-slave-6bbx krb5kdc[8403](info): set up 2 sockets
Aug 12 03:07:32 dist-test-slave-6bbx krb5kdc[8403](info): commencing operation
krb5kdc: starting...
W20260812 03:07:33.704478 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:07:33 dist-test-slave-6bbx krb5kdc[8403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:07:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:07:33.816304 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46429
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:33.927740 8429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:33.927958 8429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:33.927995 8429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:33.931780 8429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:33.931824 8429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:33.931841 8429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:33.931861 8429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:33.931877 8429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:33.936730 8429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46429
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:33.937575 8429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:33.938477 8429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:33.946539 8440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:33.946686 8429 server_base.cc:1034] running on GCE node
W20260812 03:07:33.946576 8436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:33.946552 8435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:33.947256 8429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:33.948297 8429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:33.949446 8429 hybrid_clock.cc:648] HybridClock initialized: now 1786504053949423 us; error 40 us; skew 500 ppm
Aug 12 03:07:33 dist-test-slave-6bbx krb5kdc[8403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504053, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:33.952101 8429 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:07:33.952919 8429 webserver.cc:460] Webserver started at http://127.0.0.1:46291/ using document root <none> and password file <none>
I20260812 03:07:33.953414 8429 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:33.953474 8429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:33.953670 8429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:33.955809 8429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "303af59cdfd74de08d7d28239afb2255"
format_stamp: "Formatted at 2026-08-12 03:07:33 on dist-test-slave-6bbx"
server_key: "2e64a3c7761ed9093c043d54c0b36277"
server_key_iv: "fa42b3baff0da07cd82f9826a2948c15"
server_key_version: "encryptionkey@0"
I20260812 03:07:33.956336 8429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "303af59cdfd74de08d7d28239afb2255"
format_stamp: "Formatted at 2026-08-12 03:07:33 on dist-test-slave-6bbx"
server_key: "2e64a3c7761ed9093c043d54c0b36277"
server_key_iv: "fa42b3baff0da07cd82f9826a2948c15"
server_key_version: "encryptionkey@0"
I20260812 03:07:33.959837 8429 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:07:33.962102 8447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:33.963220 8429 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:07:33.963326 8429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "303af59cdfd74de08d7d28239afb2255"
format_stamp: "Formatted at 2026-08-12 03:07:33 on dist-test-slave-6bbx"
server_key: "2e64a3c7761ed9093c043d54c0b36277"
server_key_iv: "fa42b3baff0da07cd82f9826a2948c15"
server_key_version: "encryptionkey@0"
I20260812 03:07:33.963418 8429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:33.975718 8429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:33.978518 8429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:33.978675 8429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:33.985448 8429 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46429
I20260812 03:07:33.985458 8524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46429 every 8 connection(s)
I20260812 03:07:33.986317 8429 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:33.988859 8525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:33.991335 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 8429
I20260812 03:07:33.991466 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:07:33.991832 5506 external_mini_cluster.cc:1428] Setting key 044e89ed5c34f323162e177eea99485d
I20260812 03:07:33.994477 8525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 303af59cdfd74de08d7d28239afb2255: Bootstrap starting.
I20260812 03:07:33.996842 8525 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 303af59cdfd74de08d7d28239afb2255: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:33.997627 8525 log.cc:824] T 00000000000000000000000000000000 P 303af59cdfd74de08d7d28239afb2255: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:33.999594 8525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 303af59cdfd74de08d7d28239afb2255: No bootstrap required, opened a new log
I20260812 03:07:34.000342 8533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:33.993083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:60217 (local address 127.0.0.1:46429)
0812 03:07:33.993462 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:33.993477 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:33.993510 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:07:33.994133 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:33.994150 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:33.994468 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:33.994756 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:33.995071 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:33.996093 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:33.997263 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:33.997724 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:33.997861 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:33.998692 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:33.998697 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:33.998708 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:33.999051 (+ 343us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:33.999104 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:33.999171 (+ 67us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:33.999269 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:33.999454 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 03:07:33.999576 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":100,"threads_started":1}
I20260812 03:07:34.003162 5506 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:07:34.003476 8525 raft_consensus.cc:348] T 00000000000000000000000000000000 P 303af59cdfd74de08d7d28239afb2255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "303af59cdfd74de08d7d28239afb2255" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46429 } }
I20260812 03:07:34.003752 8525 raft_consensus.cc:374] T 00000000000000000000000000000000 P 303af59cdfd74de08d7d28239afb2255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:34.003810 8525 raft_consensus.cc:729] T 00000000000000000000000000000000 P 303af59cdfd74de08d7d28239afb2255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 303af59cdfd74de08d7d28239afb2255, State: Initialized, Role: FOLLOWER
I20260812 03:07:34.004268 8525 consensus_queue.cc:260] T 00000000000000000000000000000000 P 303af59cdfd74de08d7d28239afb2255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "303af59cdfd74de08d7d28239afb2255" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46429 } }
I20260812 03:07:34.004432 8525 raft_consensus.cc:388] T 00000000000000000000000000000000 P 303af59cdfd74de08d7d28239afb2255 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:07:34.004551 8525 raft_consensus.cc:482] T 00000000000000000000000000000000 P 303af59cdfd74de08d7d28239afb2255 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:07:34.004669 8525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 303af59cdfd74de08d7d28239afb2255 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:34.005645 8525 raft_consensus.cc:504] T 00000000000000000000000000000000 P 303af59cdfd74de08d7d28239afb2255 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "303af59cdfd74de08d7d28239afb2255" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46429 } }
I20260812 03:07:34.006000 8525 leader_election.cc:302] T 00000000000000000000000000000000 P 303af59cdfd74de08d7d28239afb2255 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 303af59cdfd74de08d7d28239afb2255; no voters:
I20260812 03:07:34.006323 8525 leader_election.cc:288] T 00000000000000000000000000000000 P 303af59cdfd74de08d7d28239afb2255 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:07:34.006423 8544 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 303af59cdfd74de08d7d28239afb2255 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:34.006631 8544 raft_consensus.cc:686] T 00000000000000000000000000000000 P 303af59cdfd74de08d7d28239afb2255 [term 1 LEADER]: Becoming Leader. State: Replica: 303af59cdfd74de08d7d28239afb2255, State: Running, Role: LEADER
I20260812 03:07:34.006943 8544 consensus_queue.cc:237] T 00000000000000000000000000000000 P 303af59cdfd74de08d7d28239afb2255 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "303af59cdfd74de08d7d28239afb2255" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46429 } }
I20260812 03:07:34.007496 8525 sys_catalog.cc:564] T 00000000000000000000000000000000 P 303af59cdfd74de08d7d28239afb2255 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:34.008059 8547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 303af59cdfd74de08d7d28239afb2255 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 303af59cdfd74de08d7d28239afb2255. Latest consensus state: current_term: 1 leader_uuid: "303af59cdfd74de08d7d28239afb2255" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "303af59cdfd74de08d7d28239afb2255" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46429 } } }
I20260812 03:07:34.008193 8547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 303af59cdfd74de08d7d28239afb2255 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:34.008517 8548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 303af59cdfd74de08d7d28239afb2255 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "303af59cdfd74de08d7d28239afb2255" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "303af59cdfd74de08d7d28239afb2255" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46429 } } }
I20260812 03:07:34.008605 8548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 303af59cdfd74de08d7d28239afb2255 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:34.009816 8533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:34.005389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:35101 (local address 127.0.0.1:46429)
0812 03:07:34.005482 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:34.005486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:34.005636 (+ 150us) server_negotiation.cc:407] Connection header received
0812 03:07:34.005759 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:34.005762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:34.005807 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:34.005871 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:34.006193 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:34.006881 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:34.007983 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:34.008326 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:34.008424 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:34.009354 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:34.009358 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:34.009361 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:34.009373 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:34.009399 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:34.009411 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:34.009494 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:34.009663 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 03:07:34.009695 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260812 03:07:34.012100 8573 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 303af59cdfd74de08d7d28239afb2255: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:07:34.012163 8573 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:07:34.012233 8556 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:34.012900 8556 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:34.018056 8556 catalog_manager.cc:1347] Generated new cluster ID: 19a9de0bb48f484f9c390009e693e70f
I20260812 03:07:34.018107 8556 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:34.038509 8556 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:34.039314 8556 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:34.045619 8556 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 303af59cdfd74de08d7d28239afb2255: Generated new TSK 0
I20260812 03:07:34.046161 8556 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:07:34.053401 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 8429
W20260812 03:07:34.067788 8400 connection.cc:535] client connection to 127.0.0.1:46429 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46429): Connection reset by peer (error 104)
2026-08-12T03:07:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1501 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:07:34 dist-test-slave-6bbx krb5kdc[8581](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:07:34 dist-test-slave-6bbx krb5kdc[8581](info): set up 2 sockets
Aug 12 03:07:34 dist-test-slave-6bbx krb5kdc[8581](info): commencing operation
krb5kdc: starting...
W20260812 03:07:35.233801 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:07:35 dist-test-slave-6bbx krb5kdc[8581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:07:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:07:35.344007 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38897
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:35.449620 8599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:35.449852 8599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:35.449894 8599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:35.453740 8599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:35.453790 8599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:35.453809 8599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:35.453837 8599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:35.453863 8599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:35.458811 8599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38897
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786503790637001-5506-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:35.459734 8599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:35.460709 8599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:35.468569 8604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:35.468642 8599 server_base.cc:1034] running on GCE node
W20260812 03:07:35.468559 8610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:35.468570 8605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:35.469336 8599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:35.470329 8599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:35.471493 8599 hybrid_clock.cc:648] HybridClock initialized: now 1786504055471472 us; error 33 us; skew 500 ppm
Aug 12 03:07:35 dist-test-slave-6bbx krb5kdc[8581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504055, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:35.474051 8599 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:07:35.474875 8599 webserver.cc:460] Webserver started at http://127.0.0.1:32923/ using document root <none> and password file <none>
I20260812 03:07:35.475378 8599 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:35.475443 8599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:35.475652 8599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:35.477552 8599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "f1a575e68c294ca5bb0d69c5627e7ad8"
format_stamp: "Formatted at 2026-08-12 03:07:35 on dist-test-slave-6bbx"
server_key: "15d28045c0f573951858d5c4f20b8e38"
server_key_iv: "1377f065568088eeb931a39e9e092051"
server_key_version: "encryptionkey@0"
I20260812 03:07:35.478082 8599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "f1a575e68c294ca5bb0d69c5627e7ad8"
format_stamp: "Formatted at 2026-08-12 03:07:35 on dist-test-slave-6bbx"
server_key: "15d28045c0f573951858d5c4f20b8e38"
server_key_iv: "1377f065568088eeb931a39e9e092051"
server_key_version: "encryptionkey@0"
I20260812 03:07:35.481726 8599 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260812 03:07:35.484217 8620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:35.485371 8599 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:07:35.485491 8599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "f1a575e68c294ca5bb0d69c5627e7ad8"
format_stamp: "Formatted at 2026-08-12 03:07:35 on dist-test-slave-6bbx"
server_key: "15d28045c0f573951858d5c4f20b8e38"
server_key_iv: "1377f065568088eeb931a39e9e092051"
server_key_version: "encryptionkey@0"
I20260812 03:07:35.485585 8599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:35.490235 8599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:35.493068 8599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:35.493230 8599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:35.499951 8599 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38897
I20260812 03:07:35.499996 8689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38897 every 8 connection(s)
I20260812 03:07:35.500842 8599 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:35.503523 8692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:35.508836 8692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1a575e68c294ca5bb0d69c5627e7ad8: Bootstrap starting.
I20260812 03:07:35.509152 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 8599
I20260812 03:07:35.509274 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:07:35.509526 5506 external_mini_cluster.cc:1428] Setting key 3ff8aa6feadf59bf3272ffeed821a412
I20260812 03:07:35.511293 8692 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1a575e68c294ca5bb0d69c5627e7ad8: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:35.512040 8692 log.cc:824] T 00000000000000000000000000000000 P f1a575e68c294ca5bb0d69c5627e7ad8: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:35.514091 8692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1a575e68c294ca5bb0d69c5627e7ad8: No bootstrap required, opened a new log
I20260812 03:07:35.514482 8700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:35.510790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:47877 (local address 127.0.0.1:38897)
0812 03:07:35.511220 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:35.511234 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:35.511268 (+ 34us) server_negotiation.cc:407] Connection header received
0812 03:07:35.511932 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:35.511951 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:35.512248 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:35.512579 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:35.513321 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:35.513325 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:35.513337 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:35.513346 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:35.513395 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:35.513413 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:35.513494 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:35.513631 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 03:07:35.513785 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":128,"threads_started":1}
I20260812 03:07:35.517112 5506 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:07:35.517921 8692 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1a575e68c294ca5bb0d69c5627e7ad8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a575e68c294ca5bb0d69c5627e7ad8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38897 } }
I20260812 03:07:35.518139 8692 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1a575e68c294ca5bb0d69c5627e7ad8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:35.518200 8692 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1a575e68c294ca5bb0d69c5627e7ad8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1a575e68c294ca5bb0d69c5627e7ad8, State: Initialized, Role: FOLLOWER
I20260812 03:07:35.518620 8692 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1a575e68c294ca5bb0d69c5627e7ad8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a575e68c294ca5bb0d69c5627e7ad8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38897 } }
I20260812 03:07:35.518748 8692 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1a575e68c294ca5bb0d69c5627e7ad8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:07:35.518812 8692 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1a575e68c294ca5bb0d69c5627e7ad8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:07:35.518911 8692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1a575e68c294ca5bb0d69c5627e7ad8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:35.519842 8692 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1a575e68c294ca5bb0d69c5627e7ad8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a575e68c294ca5bb0d69c5627e7ad8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38897 } }
I20260812 03:07:35.520154 8692 leader_election.cc:302] T 00000000000000000000000000000000 P f1a575e68c294ca5bb0d69c5627e7ad8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1a575e68c294ca5bb0d69c5627e7ad8; no voters:
I20260812 03:07:35.520429 8692 leader_election.cc:288] T 00000000000000000000000000000000 P f1a575e68c294ca5bb0d69c5627e7ad8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:07:35.520500 8712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1a575e68c294ca5bb0d69c5627e7ad8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:35.520699 8712 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1a575e68c294ca5bb0d69c5627e7ad8 [term 1 LEADER]: Becoming Leader. State: Replica: f1a575e68c294ca5bb0d69c5627e7ad8, State: Running, Role: LEADER
I20260812 03:07:35.520838 8700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:35.519179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:48857 (local address 127.0.0.1:38897)
0812 03:07:35.519276 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:35.519281 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:35.519476 (+ 195us) server_negotiation.cc:407] Connection header received
0812 03:07:35.519588 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:35.519591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:35.519629 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:35.519724 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:35.520380 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:35.520383 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:35.520386 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:35.520398 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:35.520425 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:35.520437 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:35.520506 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:35.520651 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 03:07:35.520688 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 03:07:35.521030 8712 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1a575e68c294ca5bb0d69c5627e7ad8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a575e68c294ca5bb0d69c5627e7ad8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38897 } }
I20260812 03:07:35.521651 8692 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1a575e68c294ca5bb0d69c5627e7ad8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:35.522181 8712 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1a575e68c294ca5bb0d69c5627e7ad8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1a575e68c294ca5bb0d69c5627e7ad8. Latest consensus state: current_term: 1 leader_uuid: "f1a575e68c294ca5bb0d69c5627e7ad8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a575e68c294ca5bb0d69c5627e7ad8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38897 } } }
I20260812 03:07:35.522301 8712 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1a575e68c294ca5bb0d69c5627e7ad8 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:35.522548 8712 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1a575e68c294ca5bb0d69c5627e7ad8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1a575e68c294ca5bb0d69c5627e7ad8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a575e68c294ca5bb0d69c5627e7ad8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38897 } } }
I20260812 03:07:35.522687 8712 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1a575e68c294ca5bb0d69c5627e7ad8 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:35.523029 8720 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:35.525840 8720 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:35.530849 8720 catalog_manager.cc:1347] Generated new cluster ID: 5a694c037a5c44a694c65dd0bb2237d6
I20260812 03:07:35.530897 8720 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:35.546150 8720 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:35.546448 8720 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:35.556074 8720 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1a575e68c294ca5bb0d69c5627e7ad8: Generated new TSK 0
I20260812 03:07:35.556787 8720 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:07:35.564647 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 8599
2026-08-12T03:07:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1509 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:07:35 dist-test-slave-6bbx krb5kdc[8739](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:07:35 dist-test-slave-6bbx krb5kdc[8739](info): set up 2 sockets
Aug 12 03:07:35 dist-test-slave-6bbx krb5kdc[8739](info): commencing operation
krb5kdc: starting...
W20260812 03:07:36.724156 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:07:36 dist-test-slave-6bbx krb5kdc[8739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:07:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:07:36.834995 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:45365
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:45095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:45365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:36.942303 8762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:36.942538 8762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:36.942593 8762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:36.946485 8762 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:36.946537 8762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:36.946563 8762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:36.946584 8762 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:36.946609 8762 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:36.951632 8762 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:45095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:45365
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:45365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:36.952533 8762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:36.953437 8762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:36.961298 8773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:36.961331 8762 server_base.cc:1034] running on GCE node
W20260812 03:07:36.961282 8768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:36.961269 8771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:36.962045 8762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:36.963033 8762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:36.964184 8762 hybrid_clock.cc:648] HybridClock initialized: now 1786504056964172 us; error 27 us; skew 500 ppm
Aug 12 03:07:36 dist-test-slave-6bbx krb5kdc[8739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504056, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:36.966827 8762 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:07:36.967656 8762 webserver.cc:460] Webserver started at http://127.5.96.190:40143/ using document root <none> and password file <none>
I20260812 03:07:36.968181 8762 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:36.968266 8762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:36.968469 8762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:36.970325 8762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "aebc2df1d9734adc97f018c9fdfab512"
format_stamp: "Formatted at 2026-08-12 03:07:36 on dist-test-slave-6bbx"
server_key: "7fb6e77302922a368f5fd2ecc596012a"
server_key_iv: "7dcaa34a71e14498e83ec6a9172362c2"
server_key_version: "encryptionkey@0"
I20260812 03:07:36.970837 8762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "aebc2df1d9734adc97f018c9fdfab512"
format_stamp: "Formatted at 2026-08-12 03:07:36 on dist-test-slave-6bbx"
server_key: "7fb6e77302922a368f5fd2ecc596012a"
server_key_iv: "7dcaa34a71e14498e83ec6a9172362c2"
server_key_version: "encryptionkey@0"
I20260812 03:07:36.974189 8762 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 03:07:36.976346 8784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:36.977418 8762 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:07:36.977530 8762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "aebc2df1d9734adc97f018c9fdfab512"
format_stamp: "Formatted at 2026-08-12 03:07:36 on dist-test-slave-6bbx"
server_key: "7fb6e77302922a368f5fd2ecc596012a"
server_key_iv: "7dcaa34a71e14498e83ec6a9172362c2"
server_key_version: "encryptionkey@0"
I20260812 03:07:36.977617 8762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:37.000795 8762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:37.003707 8762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:37.003873 8762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:37.010638 8762 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:45365
I20260812 03:07:37.010653 8867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:45365 every 8 connection(s)
I20260812 03:07:37.011511 8762 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:37.014147 8869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:37.019351 8869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aebc2df1d9734adc97f018c9fdfab512: Bootstrap starting.
I20260812 03:07:37.020010 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 8762
I20260812 03:07:37.020135 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:07:37.020403 5506 external_mini_cluster.cc:1428] Setting key 559ccd5928b8001ca575f8c6efbc2b00
I20260812 03:07:37.021912 8869 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aebc2df1d9734adc97f018c9fdfab512: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:37.022635 8869 log.cc:824] T 00000000000000000000000000000000 P aebc2df1d9734adc97f018c9fdfab512: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:37.024602 8869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aebc2df1d9734adc97f018c9fdfab512: No bootstrap required, opened a new log
Aug 12 03:07:37 dist-test-slave-6bbx krb5kdc[8739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:07:37.028692 8869 raft_consensus.cc:348] T 00000000000000000000000000000000 P aebc2df1d9734adc97f018c9fdfab512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aebc2df1d9734adc97f018c9fdfab512" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45365 } }
I20260812 03:07:37.028880 8869 raft_consensus.cc:374] T 00000000000000000000000000000000 P aebc2df1d9734adc97f018c9fdfab512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:37.028949 8869 raft_consensus.cc:729] T 00000000000000000000000000000000 P aebc2df1d9734adc97f018c9fdfab512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aebc2df1d9734adc97f018c9fdfab512, State: Initialized, Role: FOLLOWER
I20260812 03:07:37.029423 8869 consensus_queue.cc:260] T 00000000000000000000000000000000 P aebc2df1d9734adc97f018c9fdfab512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aebc2df1d9734adc97f018c9fdfab512" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45365 } }
I20260812 03:07:37.029569 8869 raft_consensus.cc:388] T 00000000000000000000000000000000 P aebc2df1d9734adc97f018c9fdfab512 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:07:37.029677 8869 raft_consensus.cc:482] T 00000000000000000000000000000000 P aebc2df1d9734adc97f018c9fdfab512 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:07:37.029778 8869 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aebc2df1d9734adc97f018c9fdfab512 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:37.030710 8869 raft_consensus.cc:504] T 00000000000000000000000000000000 P aebc2df1d9734adc97f018c9fdfab512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aebc2df1d9734adc97f018c9fdfab512" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45365 } }
I20260812 03:07:37.031021 8869 leader_election.cc:302] T 00000000000000000000000000000000 P aebc2df1d9734adc97f018c9fdfab512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aebc2df1d9734adc97f018c9fdfab512; no voters:
I20260812 03:07:37.031292 8869 leader_election.cc:288] T 00000000000000000000000000000000 P aebc2df1d9734adc97f018c9fdfab512 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:07:37.031380 8878 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aebc2df1d9734adc97f018c9fdfab512 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:37.031571 8878 raft_consensus.cc:686] T 00000000000000000000000000000000 P aebc2df1d9734adc97f018c9fdfab512 [term 1 LEADER]: Becoming Leader. State: Replica: aebc2df1d9734adc97f018c9fdfab512, State: Running, Role: LEADER
I20260812 03:07:37.031929 8878 consensus_queue.cc:237] T 00000000000000000000000000000000 P aebc2df1d9734adc97f018c9fdfab512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aebc2df1d9734adc97f018c9fdfab512" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45365 } }
I20260812 03:07:37.032550 8869 sys_catalog.cc:564] T 00000000000000000000000000000000 P aebc2df1d9734adc97f018c9fdfab512 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:37.033509 8881 sys_catalog.cc:455] T 00000000000000000000000000000000 P aebc2df1d9734adc97f018c9fdfab512 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aebc2df1d9734adc97f018c9fdfab512" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aebc2df1d9734adc97f018c9fdfab512" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45365 } } }
I20260812 03:07:37.033824 8881 sys_catalog.cc:458] T 00000000000000000000000000000000 P aebc2df1d9734adc97f018c9fdfab512 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:37.034507 8875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:37.021425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49947 (local address 127.5.96.190:45365)
0812 03:07:37.021813 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:37.021828 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:37.021862 (+ 34us) server_negotiation.cc:407] Connection header received
0812 03:07:37.022481 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:37.022499 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:37.022849 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:37.023182 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:37.024002 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:37.024992 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:37.026146 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:37.026600 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:37.026716 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:37.029493 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:37.029498 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:37.029510 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:37.029537 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:37.031233 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:37.031817 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:37.031823 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:37.031829 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:37.031965 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:37.032344 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:37.032347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:37.032348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:37.032757 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:37.032968 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:37.033332 (+ 364us) server_negotiation.cc:299] Negotiation successful
0812 03:07:37.033564 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":106,"threads_started":1}
I20260812 03:07:37.033797 8879 sys_catalog.cc:455] T 00000000000000000000000000000000 P aebc2df1d9734adc97f018c9fdfab512 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aebc2df1d9734adc97f018c9fdfab512. Latest consensus state: current_term: 1 leader_uuid: "aebc2df1d9734adc97f018c9fdfab512" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aebc2df1d9734adc97f018c9fdfab512" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 45365 } } }
I20260812 03:07:37.035364 8879 sys_catalog.cc:458] T 00000000000000000000000000000000 P aebc2df1d9734adc97f018c9fdfab512 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:37.035113 8904 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:37.037763 5506 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:07:37.038194 8904 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:37.043255 8904 catalog_manager.cc:1347] Generated new cluster ID: f2e26de56cd144799becef59567b7413
I20260812 03:07:37.043346 8904 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:37.049990 8875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:37.041439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59577 (local address 127.5.96.190:45365)
0812 03:07:37.041535 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:37.041539 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:37.041778 (+ 239us) server_negotiation.cc:407] Connection header received
0812 03:07:37.041872 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:37.041875 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:37.041919 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:37.042009 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:37.042739 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:37.043529 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:37.044623 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:37.044972 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:37.045077 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:37.046460 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:37.046464 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:37.046467 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:37.046492 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:37.048060 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:37.048543 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:37.048546 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:37.048548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:37.048630 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:37.048899 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:37.048902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:37.048904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:37.049100 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:37.049246 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:37.049616 (+ 370us) server_negotiation.cc:299] Negotiation successful
0812 03:07:37.049841 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":38}
I20260812 03:07:37.053695 8904 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:37.054580 8904 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:37.064606 8904 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aebc2df1d9734adc97f018c9fdfab512: Generated new TSK 0
I20260812 03:07:37.065138 8904 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:07:37.072175 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 8762
W20260812 03:07:37.088758 8736 connection.cc:535] client connection to 127.5.96.190:45365 recv error: Network error: failed to read from TLS socket (remote: 127.5.96.190:45365): Connection reset by peer (error 104)
2026-08-12T03:07:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1511 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:07:37 dist-test-slave-6bbx krb5kdc[8919](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:07:37 dist-test-slave-6bbx krb5kdc[8919](info): set up 2 sockets
Aug 12 03:07:37 dist-test-slave-6bbx krb5kdc[8919](info): commencing operation
krb5kdc: starting...
W20260812 03:07:38.222710 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:07:38 dist-test-slave-6bbx krb5kdc[8919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:07:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:07:38.332542 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:35253
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:33567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:35253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:38.438421 8938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:38.438640 8938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:38.438678 8938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:38.442516 8938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:38.442567 8938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:38.442584 8938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:38.442601 8938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:38.442618 8938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:38.447544 8938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:33567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:35253
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:35253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:38.448475 8938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:38.449379 8938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:38.457456 8946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:38.457458 8949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:38.457491 8938 server_base.cc:1034] running on GCE node
W20260812 03:07:38.457496 8952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:38.458182 8938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:38.459173 8938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:38.460340 8938 hybrid_clock.cc:648] HybridClock initialized: now 1786504058460315 us; error 35 us; skew 500 ppm
Aug 12 03:07:38 dist-test-slave-6bbx krb5kdc[8919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504058, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:38.466403 8938 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:07:38.467236 8938 webserver.cc:460] Webserver started at http://127.5.96.190:38243/ using document root <none> and password file <none>
I20260812 03:07:38.467779 8938 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:38.467844 8938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:38.468040 8938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:38.469957 8938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "67d668ea5cd54c18b462067ff6576597"
format_stamp: "Formatted at 2026-08-12 03:07:38 on dist-test-slave-6bbx"
server_key: "840275eed1912bf7282112c3d77de2bd"
server_key_iv: "60769a5ef9e4e98cfb7fcd8561c83322"
server_key_version: "encryptionkey@0"
I20260812 03:07:38.470468 8938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "67d668ea5cd54c18b462067ff6576597"
format_stamp: "Formatted at 2026-08-12 03:07:38 on dist-test-slave-6bbx"
server_key: "840275eed1912bf7282112c3d77de2bd"
server_key_iv: "60769a5ef9e4e98cfb7fcd8561c83322"
server_key_version: "encryptionkey@0"
I20260812 03:07:38.473879 8938 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 03:07:38.476073 8961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:38.477178 8938 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:07:38.477288 8938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "67d668ea5cd54c18b462067ff6576597"
format_stamp: "Formatted at 2026-08-12 03:07:38 on dist-test-slave-6bbx"
server_key: "840275eed1912bf7282112c3d77de2bd"
server_key_iv: "60769a5ef9e4e98cfb7fcd8561c83322"
server_key_version: "encryptionkey@0"
I20260812 03:07:38.477376 8938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:38.503161 8938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:38.506028 8938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:38.506197 8938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:38.513021 8938 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:35253
I20260812 03:07:38.513134 9049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:35253 every 8 connection(s)
I20260812 03:07:38.513913 8938 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:38.516536 9050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:38.517498 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 8938
I20260812 03:07:38.517637 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:07:38.517953 5506 external_mini_cluster.cc:1428] Setting key ae285fc4fbbb01dd020b38e9fd57c897
I20260812 03:07:38.522372 9050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67d668ea5cd54c18b462067ff6576597: Bootstrap starting.
I20260812 03:07:38.524581 9050 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67d668ea5cd54c18b462067ff6576597: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:38.525280 9050 log.cc:824] T 00000000000000000000000000000000 P 67d668ea5cd54c18b462067ff6576597: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:38.526499 9055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:38.519292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44353 (local address 127.5.96.190:35253)
0812 03:07:38.519719 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:38.519734 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:38.519771 (+ 37us) server_negotiation.cc:407] Connection header received
0812 03:07:38.520386 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:38.520403 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:38.520714 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:38.521001 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:38.521393 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:38.522332 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:38.523242 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:38.523743 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:38.523885 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:38.524738 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:38.524744 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:38.524761 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:38.525122 (+ 361us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:38.525176 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:38.525198 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:38.525316 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:38.525543 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 03:07:38.525683 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":115,"threads_started":1}
I20260812 03:07:38.527366 9050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67d668ea5cd54c18b462067ff6576597: No bootstrap required, opened a new log
I20260812 03:07:38.529284 5506 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:07:38.531039 9050 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67d668ea5cd54c18b462067ff6576597 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d668ea5cd54c18b462067ff6576597" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35253 } }
I20260812 03:07:38.531322 9050 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67d668ea5cd54c18b462067ff6576597 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:38.531422 9050 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67d668ea5cd54c18b462067ff6576597 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67d668ea5cd54c18b462067ff6576597, State: Initialized, Role: FOLLOWER
I20260812 03:07:38.531864 9050 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67d668ea5cd54c18b462067ff6576597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d668ea5cd54c18b462067ff6576597" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35253 } }
I20260812 03:07:38.532002 9050 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67d668ea5cd54c18b462067ff6576597 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:07:38.532065 9050 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67d668ea5cd54c18b462067ff6576597 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:07:38.532159 9050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67d668ea5cd54c18b462067ff6576597 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:38.532948 9050 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67d668ea5cd54c18b462067ff6576597 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d668ea5cd54c18b462067ff6576597" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35253 } }
I20260812 03:07:38.533255 9050 leader_election.cc:302] T 00000000000000000000000000000000 P 67d668ea5cd54c18b462067ff6576597 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67d668ea5cd54c18b462067ff6576597; no voters:
I20260812 03:07:38.533520 9050 leader_election.cc:288] T 00000000000000000000000000000000 P 67d668ea5cd54c18b462067ff6576597 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:07:38.533608 9069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67d668ea5cd54c18b462067ff6576597 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:38.533787 9069 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67d668ea5cd54c18b462067ff6576597 [term 1 LEADER]: Becoming Leader. State: Replica: 67d668ea5cd54c18b462067ff6576597, State: Running, Role: LEADER
I20260812 03:07:38.534123 9069 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67d668ea5cd54c18b462067ff6576597 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d668ea5cd54c18b462067ff6576597" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35253 } }
I20260812 03:07:38.534729 9050 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67d668ea5cd54c18b462067ff6576597 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:38.535032 9055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:38.531166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37289 (local address 127.5.96.190:35253)
0812 03:07:38.531281 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:38.531285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:38.531376 (+ 91us) server_negotiation.cc:407] Connection header received
0812 03:07:38.531492 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:38.531495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:38.531535 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:38.531597 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:38.531905 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:38.532483 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:38.533308 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:38.533629 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:38.533721 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:38.534573 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:38.534576 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:38.534579 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:38.534590 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:38.534615 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:38.534625 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:38.534689 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:38.534877 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 03:07:38.534908 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:07:38.535256 9069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67d668ea5cd54c18b462067ff6576597 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67d668ea5cd54c18b462067ff6576597. Latest consensus state: current_term: 1 leader_uuid: "67d668ea5cd54c18b462067ff6576597" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d668ea5cd54c18b462067ff6576597" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35253 } } }
I20260812 03:07:38.535372 9069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67d668ea5cd54c18b462067ff6576597 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:38.535873 9070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67d668ea5cd54c18b462067ff6576597 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67d668ea5cd54c18b462067ff6576597" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d668ea5cd54c18b462067ff6576597" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35253 } } }
I20260812 03:07:38.535964 9070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67d668ea5cd54c18b462067ff6576597 [sys.catalog]: This master's current role is: LEADER
W20260812 03:07:38.539018 9097 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 67d668ea5cd54c18b462067ff6576597: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:07:38.539084 9097 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:07:38.539158 9079 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:38.540038 9079 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:38.545470 9079 catalog_manager.cc:1347] Generated new cluster ID: 3396a464d04f4726b835c23247403f32
I20260812 03:07:38.545521 9079 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:38.558228 9079 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:38.559127 9079 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:38.564661 9079 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67d668ea5cd54c18b462067ff6576597: Generated new TSK 0
I20260812 03:07:38.565207 9079 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:07:38.568481 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 8938
W20260812 03:07:38.585423 8916 connection.cc:535] client connection to 127.5.96.190:35253 recv error: Network error: failed to read from TLS socket (remote: 127.5.96.190:35253): Connection reset by peer (error 104)
2026-08-12T03:07:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1496 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:07:38 dist-test-slave-6bbx krb5kdc[9105](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:07:38 dist-test-slave-6bbx krb5kdc[9105](info): set up 2 sockets
Aug 12 03:07:38 dist-test-slave-6bbx krb5kdc[9105](info): commencing operation
krb5kdc: starting...
W20260812 03:07:39.720978 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:07:39 dist-test-slave-6bbx krb5kdc[9105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:07:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:07:39.831990 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:34581
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:33921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:34581
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:39.940730 9122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:39.940964 9122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:39.941021 9122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:39.944939 9122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:39.944993 9122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:39.945019 9122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:39.945041 9122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:39.945068 9122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:39.950034 9122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:33921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:34581
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:34581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:39.950951 9122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:39.951918 9122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:39.959755 9129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:39.959774 9131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:39.959772 9128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:39.959940 9122 server_base.cc:1034] running on GCE node
I20260812 03:07:39.960544 9122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:39.961500 9122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:39.962668 9122 hybrid_clock.cc:648] HybridClock initialized: now 1786504059962647 us; error 29 us; skew 500 ppm
Aug 12 03:07:39 dist-test-slave-6bbx krb5kdc[9105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504059, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:39.965258 9122 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:07:39.966095 9122 webserver.cc:460] Webserver started at http://127.5.96.190:39235/ using document root <none> and password file <none>
I20260812 03:07:39.966589 9122 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:39.966655 9122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:39.966856 9122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:39.968780 9122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "57d92edb21b64a2dbc159964b9709b53"
format_stamp: "Formatted at 2026-08-12 03:07:39 on dist-test-slave-6bbx"
server_key: "f875875b0da04c2130a190cbba897776"
server_key_iv: "8a5bc14fd67fbdb379dd74d03ca08d02"
server_key_version: "encryptionkey@0"
I20260812 03:07:39.969296 9122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "57d92edb21b64a2dbc159964b9709b53"
format_stamp: "Formatted at 2026-08-12 03:07:39 on dist-test-slave-6bbx"
server_key: "f875875b0da04c2130a190cbba897776"
server_key_iv: "8a5bc14fd67fbdb379dd74d03ca08d02"
server_key_version: "encryptionkey@0"
I20260812 03:07:39.972692 9122 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260812 03:07:39.974853 9141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:39.976022 9122 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 03:07:39.976133 9122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "57d92edb21b64a2dbc159964b9709b53"
format_stamp: "Formatted at 2026-08-12 03:07:39 on dist-test-slave-6bbx"
server_key: "f875875b0da04c2130a190cbba897776"
server_key_iv: "8a5bc14fd67fbdb379dd74d03ca08d02"
server_key_version: "encryptionkey@0"
I20260812 03:07:39.976220 9122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:39.980401 9122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:39.983214 9122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:39.983368 9122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:39.990062 9122 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:34581
I20260812 03:07:39.990082 9243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:34581 every 8 connection(s)
I20260812 03:07:39.990890 9122 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:39.993494 9245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:39.996958 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 9122
I20260812 03:07:39.997108 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:07:39.997438 5506 external_mini_cluster.cc:1428] Setting key d25fad71278a660b1a8bbae190a35d5c
I20260812 03:07:39.999082 9245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57d92edb21b64a2dbc159964b9709b53: Bootstrap starting.
I20260812 03:07:40.001281 9245 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57d92edb21b64a2dbc159964b9709b53: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:40.002060 9245 log.cc:824] T 00000000000000000000000000000000 P 57d92edb21b64a2dbc159964b9709b53: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:40.002316 9248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:39.998738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57935 (local address 127.5.96.190:34581)
0812 03:07:39.999143 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:39.999161 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:39.999213 (+ 52us) server_negotiation.cc:407] Connection header received
0812 03:07:39.999888 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:39.999907 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:40.000172 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:40.000474 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:40.001169 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:40.001173 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:40.001185 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:40.001195 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:40.001255 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:40.001274 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:40.001365 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:40.001489 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 03:07:40.001641 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":119,"threads_started":1}
I20260812 03:07:40.004086 9245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57d92edb21b64a2dbc159964b9709b53: No bootstrap required, opened a new log
I20260812 03:07:40.004925 5506 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:07:40.007838 9245 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57d92edb21b64a2dbc159964b9709b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57d92edb21b64a2dbc159964b9709b53" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 34581 } }
I20260812 03:07:40.008046 9245 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57d92edb21b64a2dbc159964b9709b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:40.008102 9245 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57d92edb21b64a2dbc159964b9709b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57d92edb21b64a2dbc159964b9709b53, State: Initialized, Role: FOLLOWER
I20260812 03:07:40.008275 9248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:40.006800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47229 (local address 127.5.96.190:34581)
0812 03:07:40.006898 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:40.006902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:40.007080 (+ 178us) server_negotiation.cc:407] Connection header received
0812 03:07:40.007184 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:40.007187 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:40.007217 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:40.007276 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:40.007896 (+ 620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:40.007900 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:40.007903 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:40.007914 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:40.007938 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:40.007947 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:40.008016 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:40.008130 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 03:07:40.008165 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 03:07:40.008548 9245 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57d92edb21b64a2dbc159964b9709b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57d92edb21b64a2dbc159964b9709b53" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 34581 } }
I20260812 03:07:40.008678 9245 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57d92edb21b64a2dbc159964b9709b53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:07:40.008742 9245 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57d92edb21b64a2dbc159964b9709b53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:07:40.008847 9245 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57d92edb21b64a2dbc159964b9709b53 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:40.009701 9245 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57d92edb21b64a2dbc159964b9709b53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57d92edb21b64a2dbc159964b9709b53" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 34581 } }
I20260812 03:07:40.010008 9245 leader_election.cc:302] T 00000000000000000000000000000000 P 57d92edb21b64a2dbc159964b9709b53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57d92edb21b64a2dbc159964b9709b53; no voters:
I20260812 03:07:40.010284 9245 leader_election.cc:288] T 00000000000000000000000000000000 P 57d92edb21b64a2dbc159964b9709b53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:07:40.010356 9262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57d92edb21b64a2dbc159964b9709b53 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:40.010535 9262 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57d92edb21b64a2dbc159964b9709b53 [term 1 LEADER]: Becoming Leader. State: Replica: 57d92edb21b64a2dbc159964b9709b53, State: Running, Role: LEADER
I20260812 03:07:40.010870 9262 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57d92edb21b64a2dbc159964b9709b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57d92edb21b64a2dbc159964b9709b53" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 34581 } }
I20260812 03:07:40.011349 9245 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57d92edb21b64a2dbc159964b9709b53 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:40.012383 9263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57d92edb21b64a2dbc159964b9709b53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57d92edb21b64a2dbc159964b9709b53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57d92edb21b64a2dbc159964b9709b53" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 34581 } } }
I20260812 03:07:40.012486 9263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57d92edb21b64a2dbc159964b9709b53 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:40.012799 9264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57d92edb21b64a2dbc159964b9709b53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57d92edb21b64a2dbc159964b9709b53. Latest consensus state: current_term: 1 leader_uuid: "57d92edb21b64a2dbc159964b9709b53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57d92edb21b64a2dbc159964b9709b53" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 34581 } } }
I20260812 03:07:40.012876 9272 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:40.012948 9264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57d92edb21b64a2dbc159964b9709b53 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:40.015551 9272 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:40.020576 9272 catalog_manager.cc:1347] Generated new cluster ID: 5e986cda0e104b9ebfd174724a06b092
I20260812 03:07:40.020627 9272 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:40.033759 9272 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:40.034003 9272 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:40.039544 9272 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57d92edb21b64a2dbc159964b9709b53: Generated new TSK 0
I20260812 03:07:40.040086 9272 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:07:40.045032 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 9122
2026-08-12T03:07:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1471 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:07:40 dist-test-slave-6bbx krb5kdc[9288](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:07:40 dist-test-slave-6bbx krb5kdc[9288](info): set up 2 sockets
Aug 12 03:07:40 dist-test-slave-6bbx krb5kdc[9288](info): commencing operation
krb5kdc: starting...
W20260812 03:07:41.187217 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:07:41 dist-test-slave-6bbx krb5kdc[9288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:07:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:07:41.297765 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:42803
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:43569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:42803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:41.404098 9304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:41.404325 9304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:41.404381 9304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:41.408221 9304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:41.408270 9304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:41.408298 9304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:41.408332 9304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:41.408356 9304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:41.413290 9304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:43569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:42803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:42803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:41.414172 9304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:41.415077 9304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:41.422689 9311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:41.422720 9313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:41.422720 9310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:41.422839 9304 server_base.cc:1034] running on GCE node
I20260812 03:07:41.423509 9304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:41.424507 9304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:41.425662 9304 hybrid_clock.cc:648] HybridClock initialized: now 1786504061425648 us; error 24 us; skew 500 ppm
Aug 12 03:07:41 dist-test-slave-6bbx krb5kdc[9288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504061, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:41.428243 9304 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:07:41.429008 9304 webserver.cc:460] Webserver started at http://127.5.96.190:41475/ using document root <none> and password file <none>
I20260812 03:07:41.429489 9304 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:41.429548 9304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:41.429728 9304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:41.431562 9304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "400f455ca58948e496c7331f8cad045e"
format_stamp: "Formatted at 2026-08-12 03:07:41 on dist-test-slave-6bbx"
server_key: "7d37df5a87a27b6c51bf032115646007"
server_key_iv: "75f553cee12cf178a2e70e4850941311"
server_key_version: "encryptionkey@0"
I20260812 03:07:41.432102 9304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "400f455ca58948e496c7331f8cad045e"
format_stamp: "Formatted at 2026-08-12 03:07:41 on dist-test-slave-6bbx"
server_key: "7d37df5a87a27b6c51bf032115646007"
server_key_iv: "75f553cee12cf178a2e70e4850941311"
server_key_version: "encryptionkey@0"
I20260812 03:07:41.435428 9304 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:07:41.437618 9325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:41.438761 9304 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 03:07:41.438863 9304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "400f455ca58948e496c7331f8cad045e"
format_stamp: "Formatted at 2026-08-12 03:07:41 on dist-test-slave-6bbx"
server_key: "7d37df5a87a27b6c51bf032115646007"
server_key_iv: "75f553cee12cf178a2e70e4850941311"
server_key_version: "encryptionkey@0"
I20260812 03:07:41.438952 9304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:41.457839 9304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:41.460827 9304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:41.460988 9304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:41.467537 9304 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:42803
I20260812 03:07:41.467550 9405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:42803 every 8 connection(s)
I20260812 03:07:41.468432 9304 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:41.471045 9406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:41.472883 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 9304
I20260812 03:07:41.472993 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:07:41.473259 5506 external_mini_cluster.cc:1428] Setting key 571df570ad8851467b95290b3f4e4a2d
I20260812 03:07:41.477025 9406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 400f455ca58948e496c7331f8cad045e: Bootstrap starting.
I20260812 03:07:41.479117 9406 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 400f455ca58948e496c7331f8cad045e: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:41.479969 9406 log.cc:824] T 00000000000000000000000000000000 P 400f455ca58948e496c7331f8cad045e: Log is configured to *not* fsync() on all Append() calls
Aug 12 03:07:41 dist-test-slave-6bbx krb5kdc[9288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:07:41.481838 9406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 400f455ca58948e496c7331f8cad045e: No bootstrap required, opened a new log
I20260812 03:07:41.485066 9406 raft_consensus.cc:348] T 00000000000000000000000000000000 P 400f455ca58948e496c7331f8cad045e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "400f455ca58948e496c7331f8cad045e" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42803 } }
I20260812 03:07:41.485332 9406 raft_consensus.cc:374] T 00000000000000000000000000000000 P 400f455ca58948e496c7331f8cad045e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:41.485401 9406 raft_consensus.cc:729] T 00000000000000000000000000000000 P 400f455ca58948e496c7331f8cad045e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 400f455ca58948e496c7331f8cad045e, State: Initialized, Role: FOLLOWER
I20260812 03:07:41.485795 9406 consensus_queue.cc:260] T 00000000000000000000000000000000 P 400f455ca58948e496c7331f8cad045e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "400f455ca58948e496c7331f8cad045e" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42803 } }
I20260812 03:07:41.485920 9406 raft_consensus.cc:388] T 00000000000000000000000000000000 P 400f455ca58948e496c7331f8cad045e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:07:41.485984 9406 raft_consensus.cc:482] T 00000000000000000000000000000000 P 400f455ca58948e496c7331f8cad045e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:07:41.486037 9409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:41.474478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53377 (local address 127.5.96.190:42803)
0812 03:07:41.474885 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:41.474899 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:41.474932 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:07:41.475554 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:41.475570 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:41.475923 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:41.476207 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:41.476989 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:41.477945 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:41.478767 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:41.479212 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:41.479363 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:41.481835 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:41.481841 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:41.481852 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:41.481876 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:41.483365 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:41.483800 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:41.483804 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:41.483809 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:41.483917 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:41.484167 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:41.484171 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:41.484172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:41.484522 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:41.484707 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:41.484971 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 03:07:41.485231 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":142,"threads_started":1}
I20260812 03:07:41.486078 9406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 400f455ca58948e496c7331f8cad045e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:41.487062 9406 raft_consensus.cc:504] T 00000000000000000000000000000000 P 400f455ca58948e496c7331f8cad045e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "400f455ca58948e496c7331f8cad045e" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42803 } }
I20260812 03:07:41.487382 9406 leader_election.cc:302] T 00000000000000000000000000000000 P 400f455ca58948e496c7331f8cad045e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 400f455ca58948e496c7331f8cad045e; no voters:
I20260812 03:07:41.487653 9406 leader_election.cc:288] T 00000000000000000000000000000000 P 400f455ca58948e496c7331f8cad045e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:07:41.487771 9411 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 400f455ca58948e496c7331f8cad045e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:41.487962 9411 raft_consensus.cc:686] T 00000000000000000000000000000000 P 400f455ca58948e496c7331f8cad045e [term 1 LEADER]: Becoming Leader. State: Replica: 400f455ca58948e496c7331f8cad045e, State: Running, Role: LEADER
I20260812 03:07:41.488293 9411 consensus_queue.cc:237] T 00000000000000000000000000000000 P 400f455ca58948e496c7331f8cad045e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "400f455ca58948e496c7331f8cad045e" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42803 } }
I20260812 03:07:41.488848 5506 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:07:41.488898 9406 sys_catalog.cc:564] T 00000000000000000000000000000000 P 400f455ca58948e496c7331f8cad045e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:41.489476 9411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 400f455ca58948e496c7331f8cad045e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 400f455ca58948e496c7331f8cad045e. Latest consensus state: current_term: 1 leader_uuid: "400f455ca58948e496c7331f8cad045e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "400f455ca58948e496c7331f8cad045e" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42803 } } }
I20260812 03:07:41.489651 9411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 400f455ca58948e496c7331f8cad045e [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:41.489874 9413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 400f455ca58948e496c7331f8cad045e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "400f455ca58948e496c7331f8cad045e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "400f455ca58948e496c7331f8cad045e" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 42803 } } }
I20260812 03:07:41.489979 9413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 400f455ca58948e496c7331f8cad045e [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:41.490067 9428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:41.493242 9428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:41.498708 9428 catalog_manager.cc:1347] Generated new cluster ID: 34b52c0d62f143a2b738f52838c89841
I20260812 03:07:41.498809 9428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:41.503455 9409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:41.492271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47157 (local address 127.5.96.190:42803)
0812 03:07:41.492388 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:41.492392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:41.495519 (+ 3127us) server_negotiation.cc:407] Connection header received
0812 03:07:41.495627 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:41.495630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:41.495694 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:41.495766 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:41.496567 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:41.497333 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:41.498363 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:41.498706 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:41.498815 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:41.500413 (+ 1598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:41.500417 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:41.500419 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:41.500442 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:41.501930 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:41.502385 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:41.502388 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:41.502390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:41.502461 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:41.502713 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:41.502715 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:41.502717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:41.502882 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:41.502988 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:41.503211 (+ 223us) server_negotiation.cc:299] Negotiation successful
0812 03:07:41.503315 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 03:07:41.514325 9428 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:41.515404 9428 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:41.530161 9428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 400f455ca58948e496c7331f8cad045e: Generated new TSK 0
I20260812 03:07:41.530856 9428 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:07:41.538745 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 9304
2026-08-12T03:07:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1498 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:07:41 dist-test-slave-6bbx krb5kdc[9453](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:07:41 dist-test-slave-6bbx krb5kdc[9453](info): set up 2 sockets
Aug 12 03:07:41 dist-test-slave-6bbx krb5kdc[9453](info): commencing operation
krb5kdc: starting...
W20260812 03:07:42.693547 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:07:42 dist-test-slave-6bbx krb5kdc[9453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:07:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:07:42.803694 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:39343
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:44033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:39343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:42.913110 9478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:42.913336 9478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:42.913374 9478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:42.917227 9478 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:42.917275 9478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:42.917292 9478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:42.917312 9478 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:42.917330 9478 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:42.922271 9478 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:44033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:39343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:39343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:42.923135 9478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:42.924063 9478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:42.931910 9484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:42.931922 9490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:42.932009 9478 server_base.cc:1034] running on GCE node
W20260812 03:07:42.931922 9486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:42.932651 9478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:42.933619 9478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:42.934778 9478 hybrid_clock.cc:648] HybridClock initialized: now 1786504062934765 us; error 25 us; skew 500 ppm
Aug 12 03:07:42 dist-test-slave-6bbx krb5kdc[9453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504062, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:42.937297 9478 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:07:42.938112 9478 webserver.cc:460] Webserver started at http://127.5.96.190:32887/ using document root <none> and password file <none>
I20260812 03:07:42.938606 9478 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:42.938668 9478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:42.938853 9478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:42.940826 9478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "268c5e71c9bf459f8a34e5bd466abe8c"
format_stamp: "Formatted at 2026-08-12 03:07:42 on dist-test-slave-6bbx"
server_key: "fa32720ddeae3ee2f3c72c0d736cd2b7"
server_key_iv: "a284a86324231a34afd8bf58dcf9c0c6"
server_key_version: "encryptionkey@0"
I20260812 03:07:42.941344 9478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "268c5e71c9bf459f8a34e5bd466abe8c"
format_stamp: "Formatted at 2026-08-12 03:07:42 on dist-test-slave-6bbx"
server_key: "fa32720ddeae3ee2f3c72c0d736cd2b7"
server_key_iv: "a284a86324231a34afd8bf58dcf9c0c6"
server_key_version: "encryptionkey@0"
I20260812 03:07:42.944825 9478 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:07:42.946934 9498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:42.948091 9478 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 03:07:42.948199 9478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "268c5e71c9bf459f8a34e5bd466abe8c"
format_stamp: "Formatted at 2026-08-12 03:07:42 on dist-test-slave-6bbx"
server_key: "fa32720ddeae3ee2f3c72c0d736cd2b7"
server_key_iv: "a284a86324231a34afd8bf58dcf9c0c6"
server_key_version: "encryptionkey@0"
I20260812 03:07:42.948284 9478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:42.965289 9478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:42.967622 9478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:42.967813 9478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:42.974154 9478 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:39343
I20260812 03:07:42.974160 9578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:39343 every 8 connection(s)
I20260812 03:07:42.974998 9478 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:42.977631 9580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:42.978540 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 9478
I20260812 03:07:42.978682 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:07:42.978978 5506 external_mini_cluster.cc:1428] Setting key d0185827f48414c8d9ed06275946f89d
I20260812 03:07:42.983538 9580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 268c5e71c9bf459f8a34e5bd466abe8c: Bootstrap starting.
I20260812 03:07:42.985989 9580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 268c5e71c9bf459f8a34e5bd466abe8c: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:42.986713 9580 log.cc:824] T 00000000000000000000000000000000 P 268c5e71c9bf459f8a34e5bd466abe8c: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:42.987231 9583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:42.980217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51037 (local address 127.5.96.190:39343)
0812 03:07:42.980588 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:42.980603 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:42.980637 (+ 34us) server_negotiation.cc:407] Connection header received
0812 03:07:42.981260 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:42.981276 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:42.981583 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:42.981843 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:42.982185 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:42.983118 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:42.984328 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:42.984776 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:42.984884 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:42.985618 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:42.985624 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:42.985636 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:42.985996 (+ 360us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:42.986052 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:42.986084 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:42.986177 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:42.986322 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 03:07:42.986514 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":91,"threads_started":1}
I20260812 03:07:42.988601 9580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 268c5e71c9bf459f8a34e5bd466abe8c: No bootstrap required, opened a new log
I20260812 03:07:42.990041 5506 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:07:42.993140 9580 raft_consensus.cc:348] T 00000000000000000000000000000000 P 268c5e71c9bf459f8a34e5bd466abe8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "268c5e71c9bf459f8a34e5bd466abe8c" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39343 } }
I20260812 03:07:42.993355 9580 raft_consensus.cc:374] T 00000000000000000000000000000000 P 268c5e71c9bf459f8a34e5bd466abe8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:42.993410 9580 raft_consensus.cc:729] T 00000000000000000000000000000000 P 268c5e71c9bf459f8a34e5bd466abe8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 268c5e71c9bf459f8a34e5bd466abe8c, State: Initialized, Role: FOLLOWER
I20260812 03:07:42.993855 9580 consensus_queue.cc:260] T 00000000000000000000000000000000 P 268c5e71c9bf459f8a34e5bd466abe8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "268c5e71c9bf459f8a34e5bd466abe8c" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39343 } }
I20260812 03:07:42.994014 9580 raft_consensus.cc:388] T 00000000000000000000000000000000 P 268c5e71c9bf459f8a34e5bd466abe8c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:07:42.994073 9580 raft_consensus.cc:482] T 00000000000000000000000000000000 P 268c5e71c9bf459f8a34e5bd466abe8c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:07:42.994160 9580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 268c5e71c9bf459f8a34e5bd466abe8c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:42.995121 9580 raft_consensus.cc:504] T 00000000000000000000000000000000 P 268c5e71c9bf459f8a34e5bd466abe8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "268c5e71c9bf459f8a34e5bd466abe8c" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39343 } }
I20260812 03:07:42.995476 9580 leader_election.cc:302] T 00000000000000000000000000000000 P 268c5e71c9bf459f8a34e5bd466abe8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 268c5e71c9bf459f8a34e5bd466abe8c; no voters:
I20260812 03:07:42.995798 9580 leader_election.cc:288] T 00000000000000000000000000000000 P 268c5e71c9bf459f8a34e5bd466abe8c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:07:42.995879 9591 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 268c5e71c9bf459f8a34e5bd466abe8c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:42.995985 9583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:42.992235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37541 (local address 127.5.96.190:39343)
0812 03:07:42.992347 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:42.992351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:42.992492 (+ 141us) server_negotiation.cc:407] Connection header received
0812 03:07:42.992607 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:42.992609 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:42.992642 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:42.992695 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:42.992956 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:42.993446 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:42.994461 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:42.994702 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:42.994781 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:42.995503 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:42.995506 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:42.995508 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:42.995518 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:42.995541 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:42.995551 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:42.995618 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:42.995754 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 03:07:42.995801 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:07:42.996110 9591 raft_consensus.cc:686] T 00000000000000000000000000000000 P 268c5e71c9bf459f8a34e5bd466abe8c [term 1 LEADER]: Becoming Leader. State: Replica: 268c5e71c9bf459f8a34e5bd466abe8c, State: Running, Role: LEADER
I20260812 03:07:42.996408 9591 consensus_queue.cc:237] T 00000000000000000000000000000000 P 268c5e71c9bf459f8a34e5bd466abe8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "268c5e71c9bf459f8a34e5bd466abe8c" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39343 } }
I20260812 03:07:42.997049 9580 sys_catalog.cc:564] T 00000000000000000000000000000000 P 268c5e71c9bf459f8a34e5bd466abe8c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:42.997524 9591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 268c5e71c9bf459f8a34e5bd466abe8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "268c5e71c9bf459f8a34e5bd466abe8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "268c5e71c9bf459f8a34e5bd466abe8c" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39343 } } }
I20260812 03:07:42.997673 9591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 268c5e71c9bf459f8a34e5bd466abe8c [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:42.997934 9591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 268c5e71c9bf459f8a34e5bd466abe8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 268c5e71c9bf459f8a34e5bd466abe8c. Latest consensus state: current_term: 1 leader_uuid: "268c5e71c9bf459f8a34e5bd466abe8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "268c5e71c9bf459f8a34e5bd466abe8c" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39343 } } }
I20260812 03:07:42.998075 9591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 268c5e71c9bf459f8a34e5bd466abe8c [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:42.998440 9600 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:43.001482 9600 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:43.006592 9600 catalog_manager.cc:1347] Generated new cluster ID: 4f1881fee67d4a5995a09b0c4de35945
I20260812 03:07:43.006654 9600 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:43.019558 9600 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:43.020741 9600 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:43.026464 9600 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 268c5e71c9bf459f8a34e5bd466abe8c: Generated new TSK 0
I20260812 03:07:43.027040 9600 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:07:43.035924 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 9478
2026-08-12T03:07:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1490 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:07:43 dist-test-slave-6bbx krb5kdc[9623](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:07:43 dist-test-slave-6bbx krb5kdc[9623](info): set up 2 sockets
Aug 12 03:07:43 dist-test-slave-6bbx krb5kdc[9623](info): commencing operation
krb5kdc: starting...
W20260812 03:07:44.181265 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:07:44 dist-test-slave-6bbx krb5kdc[9623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:07:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:07:44.291859 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:39169
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:35677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:39169
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:44.399356 9647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:44.399586 9647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:44.399642 9647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:44.403504 9647 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:44.403558 9647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:44.403582 9647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:44.403615 9647 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:44.403649 9647 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:44.408643 9647 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:35677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:39169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:39169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:44.409541 9647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:44.410462 9647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:44.418388 9654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:44.418412 9657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:44.418439 9655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:44.418601 9647 server_base.cc:1034] running on GCE node
I20260812 03:07:44.419106 9647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:44.420102 9647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:44.421262 9647 hybrid_clock.cc:648] HybridClock initialized: now 1786504064421240 us; error 29 us; skew 500 ppm
Aug 12 03:07:44 dist-test-slave-6bbx krb5kdc[9623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504064, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:44.427824 9647 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:07:44.428579 9647 webserver.cc:460] Webserver started at http://127.5.96.190:46325/ using document root <none> and password file <none>
I20260812 03:07:44.429054 9647 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:44.429108 9647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:44.429267 9647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:44.431092 9647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "86b7730a92344332bb61e8eaf228e88b"
format_stamp: "Formatted at 2026-08-12 03:07:44 on dist-test-slave-6bbx"
server_key: "f85552bcabcb8e0dfe2df656a0a7ae99"
server_key_iv: "176782058ce4644eae441768f2ba576e"
server_key_version: "encryptionkey@0"
I20260812 03:07:44.431568 9647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "86b7730a92344332bb61e8eaf228e88b"
format_stamp: "Formatted at 2026-08-12 03:07:44 on dist-test-slave-6bbx"
server_key: "f85552bcabcb8e0dfe2df656a0a7ae99"
server_key_iv: "176782058ce4644eae441768f2ba576e"
server_key_version: "encryptionkey@0"
I20260812 03:07:44.434922 9647 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:07:44.437115 9668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:44.438249 9647 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:07:44.438346 9647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "86b7730a92344332bb61e8eaf228e88b"
format_stamp: "Formatted at 2026-08-12 03:07:44 on dist-test-slave-6bbx"
server_key: "f85552bcabcb8e0dfe2df656a0a7ae99"
server_key_iv: "176782058ce4644eae441768f2ba576e"
server_key_version: "encryptionkey@0"
I20260812 03:07:44.438422 9647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:44.442603 9647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:44.445372 9647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:44.445504 9647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:44.451809 9647 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:39169
I20260812 03:07:44.451843 9743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:39169 every 8 connection(s)
I20260812 03:07:44.452613 9647 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:44.455118 9745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:44.456847 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 9647
I20260812 03:07:44.456977 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:07:44.457211 5506 external_mini_cluster.cc:1428] Setting key d27f789681e1a427d407dc7c8a8d84b3
I20260812 03:07:44.461225 9745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86b7730a92344332bb61e8eaf228e88b: Bootstrap starting.
I20260812 03:07:44.462028 9749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:44.458455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35639 (local address 127.5.96.190:39169)
0812 03:07:44.458861 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:44.458874 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:44.458907 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:07:44.459541 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:44.459558 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:44.459912 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:44.460219 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:44.460954 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:44.460958 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:44.460970 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:44.460979 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:44.461030 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:44.461048 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:44.461169 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:44.461267 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 03:07:44.461412 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":124,"threads_started":1}
I20260812 03:07:44.463405 9745 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86b7730a92344332bb61e8eaf228e88b: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:44.464208 9745 log.cc:824] T 00000000000000000000000000000000 P 86b7730a92344332bb61e8eaf228e88b: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:44.464505 5506 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:07:44.466013 9745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86b7730a92344332bb61e8eaf228e88b: No bootstrap required, opened a new log
I20260812 03:07:44.467865 9749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:44.466428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36059 (local address 127.5.96.190:39169)
0812 03:07:44.466521 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:44.466525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:44.466697 (+ 172us) server_negotiation.cc:407] Connection header received
0812 03:07:44.466832 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:44.466835 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:44.466864 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:44.466920 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:44.467502 (+ 582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:44.467506 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:44.467509 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:07:44.467520 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:44.467544 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:07:44.467554 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:44.467621 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:44.467730 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 03:07:44.467764 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 03:07:44.469678 9745 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86b7730a92344332bb61e8eaf228e88b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86b7730a92344332bb61e8eaf228e88b" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39169 } }
I20260812 03:07:44.469907 9745 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86b7730a92344332bb61e8eaf228e88b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:44.469967 9745 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86b7730a92344332bb61e8eaf228e88b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86b7730a92344332bb61e8eaf228e88b, State: Initialized, Role: FOLLOWER
I20260812 03:07:44.470448 9745 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86b7730a92344332bb61e8eaf228e88b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86b7730a92344332bb61e8eaf228e88b" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39169 } }
I20260812 03:07:44.470580 9745 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86b7730a92344332bb61e8eaf228e88b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:07:44.470625 9745 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86b7730a92344332bb61e8eaf228e88b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:07:44.470690 9745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86b7730a92344332bb61e8eaf228e88b [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:44.471423 9745 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86b7730a92344332bb61e8eaf228e88b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86b7730a92344332bb61e8eaf228e88b" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39169 } }
I20260812 03:07:44.471735 9745 leader_election.cc:302] T 00000000000000000000000000000000 P 86b7730a92344332bb61e8eaf228e88b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86b7730a92344332bb61e8eaf228e88b; no voters:
I20260812 03:07:44.471971 9745 leader_election.cc:288] T 00000000000000000000000000000000 P 86b7730a92344332bb61e8eaf228e88b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:07:44.472064 9763 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86b7730a92344332bb61e8eaf228e88b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:44.472257 9763 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86b7730a92344332bb61e8eaf228e88b [term 1 LEADER]: Becoming Leader. State: Replica: 86b7730a92344332bb61e8eaf228e88b, State: Running, Role: LEADER
I20260812 03:07:44.472589 9763 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86b7730a92344332bb61e8eaf228e88b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86b7730a92344332bb61e8eaf228e88b" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39169 } }
I20260812 03:07:44.473008 9745 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86b7730a92344332bb61e8eaf228e88b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:44.474027 9765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86b7730a92344332bb61e8eaf228e88b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86b7730a92344332bb61e8eaf228e88b. Latest consensus state: current_term: 1 leader_uuid: "86b7730a92344332bb61e8eaf228e88b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86b7730a92344332bb61e8eaf228e88b" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39169 } } }
I20260812 03:07:44.474156 9765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86b7730a92344332bb61e8eaf228e88b [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:44.474557 9780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:44.474525 9764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86b7730a92344332bb61e8eaf228e88b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86b7730a92344332bb61e8eaf228e88b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86b7730a92344332bb61e8eaf228e88b" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39169 } } }
I20260812 03:07:44.475140 9764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86b7730a92344332bb61e8eaf228e88b [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:44.477536 9780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:44.482249 9780 catalog_manager.cc:1347] Generated new cluster ID: 263271ad5dce4d378bb3577457937070
I20260812 03:07:44.482309 9780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:44.501188 9780 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:44.501479 9780 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:44.506590 9780 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86b7730a92344332bb61e8eaf228e88b: Generated new TSK 0
I20260812 03:07:44.507283 9780 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:07:44.511201 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 9647
2026-08-12T03:07:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1484 ms)
[----------] 18 tests from AuthTokenIssuingTest (26955 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:07:44 dist-test-slave-6bbx krb5kdc[9800](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:07:44 dist-test-slave-6bbx krb5kdc[9800](info): set up 2 sockets
Aug 12 03:07:44 dist-test-slave-6bbx krb5kdc[9800](info): commencing operation
krb5kdc: starting...
W20260812 03:07:45.675520 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:07:45 dist-test-slave-6bbx krb5kdc[9800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:07:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:07:45.786371 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:39919
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:34271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:39919,127.5.96.189:36101,127.5.96.188:33307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:45.896716 9814 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:07:45.896973 9814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:45.896996 9814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:45.897054 9814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:45.900979 9814 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:45.901029 9814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:45.901054 9814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:45.901078 9814 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:45.901118 9814 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:45.907450 9814 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:34271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:39919,127.5.96.189:36101,127.5.96.188:33307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:39919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:45.908545 9814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:45.909538 9814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:45.917881 9821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:45.917857 9823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:45.917928 9814 server_base.cc:1034] running on GCE node
W20260812 03:07:45.917881 9825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:45.918622 9814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:45.919618 9814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:45.920789 9814 hybrid_clock.cc:648] HybridClock initialized: now 1786504065920769 us; error 34 us; skew 500 ppm
Aug 12 03:07:45 dist-test-slave-6bbx krb5kdc[9800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504065, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:45.923305 9814 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:07:45.924175 9814 webserver.cc:460] Webserver started at http://127.5.96.190:38359/ using document root <none> and password file <none>
I20260812 03:07:45.924700 9814 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:45.924775 9814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:45.924979 9814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:45.926911 9814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "d0f2939d0c5348028f0b1d1502602793"
format_stamp: "Formatted at 2026-08-12 03:07:45 on dist-test-slave-6bbx"
server_key: "e0999187d8137ee9e8cb84046b0f31f7"
server_key_iv: "af09dd10ba7cd9e9cec24ba5537de722"
server_key_version: "encryptionkey@0"
I20260812 03:07:45.927424 9814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "d0f2939d0c5348028f0b1d1502602793"
format_stamp: "Formatted at 2026-08-12 03:07:45 on dist-test-slave-6bbx"
server_key: "e0999187d8137ee9e8cb84046b0f31f7"
server_key_iv: "af09dd10ba7cd9e9cec24ba5537de722"
server_key_version: "encryptionkey@0"
I20260812 03:07:45.930925 9814 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 03:07:45.933171 9835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:45.934268 9814 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:07:45.934387 9814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "d0f2939d0c5348028f0b1d1502602793"
format_stamp: "Formatted at 2026-08-12 03:07:45 on dist-test-slave-6bbx"
server_key: "e0999187d8137ee9e8cb84046b0f31f7"
server_key_iv: "af09dd10ba7cd9e9cec24ba5537de722"
server_key_version: "encryptionkey@0"
I20260812 03:07:45.934474 9814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:45.947497 9814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:45.949971 9814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:45.950132 9814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:45.956787 9814 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:39919
I20260812 03:07:45.957013 9908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:39919 every 8 connection(s)
I20260812 03:07:45.957664 9814 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:45.960261 9910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:45.961755 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 9814
I20260812 03:07:45.961884 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:07:45.962123 5506 external_mini_cluster.cc:1428] Setting key cab3bbadf23954c3c2e1ae2e41251bdd
I20260812 03:07:45.963896 9910 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39919 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.96.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.189@KRBTEST.COM" created.
Aug 12 03:07:45 dist-test-slave-6bbx krb5kdc[9800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504065, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:07:45.978667 9919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:45.966030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39330 (local address 127.5.96.190:39919)
0812 03:07:45.966413 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:45.966432 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:45.966672 (+ 240us) server_negotiation.cc:407] Connection header received
0812 03:07:45.967639 (+ 967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:45.967644 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:45.967860 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:45.968020 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:45.968840 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:45.969683 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:45.970622 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:45.971078 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:45.971201 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:45.974034 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:45.974040 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:45.974049 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:45.974072 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:45.975828 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:45.976350 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:45.976354 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:45.976355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:45.976455 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:45.976809 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:45.976812 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:45.976813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:45.977159 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:45.977261 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:45.977684 (+ 423us) server_negotiation.cc:299] Negotiation successful
0812 03:07:45.977884 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":251,"thread_start_us":97,"threads_started":1}
I20260812 03:07:45.978670 9920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:45.966031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39919 (local address 127.0.0.1:39330)
0812 03:07:45.966539 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:45.966574 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:45.967357 (+ 783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:45.968032 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:45.968037 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:45.968452 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:45.968687 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:45.968700 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:45.969801 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:45.969803 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:45.970488 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:45.970497 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:45.971209 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:45.971211 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:45.971309 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:45.971851 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:45.971867 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:45.973846 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:45.975933 (+ 2087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:45.975943 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:45.975953 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:45.976245 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:45.976550 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:45.976553 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:45.976556 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:45.976711 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:45.977282 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:45.977288 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:45.977445 (+ 157us) client_negotiation.cc:769] Sending connection context
0812 03:07:45.977632 (+ 187us) client_negotiation.cc:240] Negotiation successful
0812 03:07:45.977953 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":54,"thread_start_us":155,"threads_started":1}
I20260812 03:07:45.980816 9910 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.189" port: 36101 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:07:45.981437 9920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:45.981196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:36101 (local address 127.0.0.1:46622)
0812 03:07:45.981312 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:45.981375 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.189:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 03:07:45.981606 9836 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.96.189:36101: connect: Connection refused (error 111)
I20260812 03:07:45.982363 9920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:45.982144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:36101 (local address 127.0.0.1:46632)
0812 03:07:45.982245 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:45.982301 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.189:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 03:07:45.982537 9910 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.96.189:36101: Network error: Client connection negotiation failed: client connection to 127.5.96.189:36101: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.189.keytab.
Entry for principal kudu/127.5.96.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.189.keytab.
I20260812 03:07:45.987022 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.189:36101
--webserver_interface=127.5.96.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.189.keytab
--principal=kudu/127.5.96.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:34271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:39919,127.5.96.189:36101,127.5.96.188:33307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:07:46.031747 9910 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.96.189" port: 36101 } attempt: 1
I20260812 03:07:46.032584 9920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.032270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:36101 (local address 127.0.0.1:46646)
0812 03:07:46.032426 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:46.032495 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.189:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 03:07:46.033243 9920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.033031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:36101 (local address 127.0.0.1:46658)
0812 03:07:46.033125 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:46.033175 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.189:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 03:07:46.033380 9910 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.96.189:36101: Network error: Client connection negotiation failed: client connection to 127.5.96.189:36101: connect: Connection refused (error 111)
W20260812 03:07:46.095167 9927 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:07:46.095427 9927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:46.095464 9927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:46.095520 9927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:46.099359 9927 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:46.099406 9927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:46.099424 9927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:46.099452 9927 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:46.099478 9927 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:46.101536 9910 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.96.189" port: 36101 } attempt: 2
I20260812 03:07:46.102365 9920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.102043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:36101 (local address 127.0.0.1:46668)
0812 03:07:46.102205 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:46.102278 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.189:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 03:07:46.102998 9920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.102798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:36101 (local address 127.0.0.1:46672)
0812 03:07:46.102887 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:46.102934 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.189:36101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 03:07:46.103111 9910 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.96.189:36101: Network error: Client connection negotiation failed: client connection to 127.5.96.189:36101: connect: Connection refused (error 111)
I20260812 03:07:46.104892 9927 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:34271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:39919,127.5.96.189:36101,127.5.96.188:33307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.189.keytab
--principal=kudu/127.5.96.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.189:36101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.96.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:46.105772 9927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:46.106647 9927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:46.114368 9933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:46.114378 9932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:46.114380 9936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:46.114773 9927 server_base.cc:1034] running on GCE node
I20260812 03:07:46.115267 9927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:46.116261 9927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:46.117429 9927 hybrid_clock.cc:648] HybridClock initialized: now 1786504066117405 us; error 37 us; skew 500 ppm
Aug 12 03:07:46 dist-test-slave-6bbx krb5kdc[9800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504066, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:46.119951 9927 init.cc:377] Logged in from keytab as kudu/127.5.96.189@KRBTEST.COM (short username kudu)
I20260812 03:07:46.120774 9927 webserver.cc:460] Webserver started at http://127.5.96.189:38237/ using document root <none> and password file <none>
I20260812 03:07:46.121260 9927 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:46.121323 9927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:46.121513 9927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:46.123422 9927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-1/data/instance:
uuid: "24d7b7a09067448fb0994cd0f2f6231f"
format_stamp: "Formatted at 2026-08-12 03:07:46 on dist-test-slave-6bbx"
server_key: "24298513217e0a674efc61e6778c0b1b"
server_key_iv: "b6a8bc4fc9c8a5cc42ce622c0ed989fd"
server_key_version: "encryptionkey@0"
I20260812 03:07:46.123988 9927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-1/wal/instance:
uuid: "24d7b7a09067448fb0994cd0f2f6231f"
format_stamp: "Formatted at 2026-08-12 03:07:46 on dist-test-slave-6bbx"
server_key: "24298513217e0a674efc61e6778c0b1b"
server_key_iv: "b6a8bc4fc9c8a5cc42ce622c0ed989fd"
server_key_version: "encryptionkey@0"
I20260812 03:07:46.127418 9927 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260812 03:07:46.129694 9944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:46.130820 9927 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:07:46.130930 9927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-1/wal
uuid: "24d7b7a09067448fb0994cd0f2f6231f"
format_stamp: "Formatted at 2026-08-12 03:07:46 on dist-test-slave-6bbx"
server_key: "24298513217e0a674efc61e6778c0b1b"
server_key_iv: "b6a8bc4fc9c8a5cc42ce622c0ed989fd"
server_key_version: "encryptionkey@0"
I20260812 03:07:46.131016 9927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:46.143373 9927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:46.145849 9927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:46.146004 9927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:46.152364 9927 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.189:36101
I20260812 03:07:46.152634 10022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.189:36101 every 8 connection(s)
I20260812 03:07:46.153199 9927 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-1/data/info.pb
I20260812 03:07:46.155738 10025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:46.159125 10025 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39919 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:07:46.162611 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 9927
I20260812 03:07:46.162726 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-1/wal/instance
I20260812 03:07:46.162961 5506 external_mini_cluster.cc:1428] Setting key 0e03af390b54204d64d64bcc5da62131
Aug 12 03:07:46 dist-test-slave-6bbx krb5kdc[9800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504066, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.189@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.96.188@KRBTEST.COM; defaulting to no policy
I20260812 03:07:46.172071 9919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.160759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39332 (local address 127.5.96.190:39919)
0812 03:07:46.160879 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:46.160882 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:46.161559 (+ 677us) server_negotiation.cc:407] Connection header received
0812 03:07:46.162441 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:46.162445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:46.162503 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:46.162574 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:46.163703 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.164215 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.165342 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.165577 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.165668 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:46.168172 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:46.168175 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:46.168177 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:46.168201 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:46.170022 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.170591 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.170593 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.170594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.170660 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.171002 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.171004 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.171005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.171156 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:46.171224 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:46.171819 (+ 595us) server_negotiation.cc:299] Negotiation successful
0812 03:07:46.171952 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:07:46.172794 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.161020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39919 (local address 127.0.0.1:39332)
0812 03:07:46.161420 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:46.161460 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:46.162219 (+ 759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:46.162756 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:46.162764 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:46.163102 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:46.163542 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:46.163554 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.164336 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.164340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:46.165213 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:46.165235 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.165714 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.165718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:46.165875 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:46.166360 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:46.166377 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:46.168013 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:46.170140 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:46.170146 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:46.170160 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:46.170504 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:46.170738 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:46.170741 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:46.170744 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:46.170920 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:46.171265 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:46.171272 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:46.171564 (+ 292us) client_negotiation.cc:769] Sending connection context
0812 03:07:46.171780 (+ 216us) client_negotiation.cc:240] Negotiation successful
0812 03:07:46.172086 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":98,"threads_started":1}
I20260812 03:07:46.173956 10025 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.189" port: 36101 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.188@KRBTEST.COM" created.
Aug 12 03:07:46 dist-test-slave-6bbx krb5kdc[9800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504066, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.189@KRBTEST.COM for kudu/127.5.96.189@KRBTEST.COM
I20260812 03:07:46.182389 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.174368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:36101 (local address 127.0.0.1:46682)
0812 03:07:46.174483 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:46.174495 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:46.174569 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:46.175124 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:46.175126 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:46.175244 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:46.175323 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:46.175328 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.176351 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.176354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:46.177130 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:46.177141 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.177666 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.177670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:46.177787 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:46.178159 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:46.178173 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:46.179403 (+ 1230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:46.181059 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:46.181063 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:46.181065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:46.181291 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:46.181530 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:46.181532 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:46.181534 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:46.181623 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:46.181995 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:46.181997 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:46.182070 (+ 73us) client_negotiation.cc:769] Sending connection context
0812 03:07:46.182162 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 03:07:46.182279 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 03:07:46.182528 10031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.174471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46682 (local address 127.5.96.189:36101)
0812 03:07:46.174756 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:46.174767 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:46.174789 (+ 22us) server_negotiation.cc:407] Connection header received
0812 03:07:46.174829 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:46.174834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:46.175011 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:46.175142 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:46.175418 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.176242 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.177238 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.177553 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.177627 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:46.179531 (+ 1904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:46.179540 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:46.179549 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:46.179569 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:46.180954 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.181389 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.181393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.181394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.181460 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.181701 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.181703 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.181704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.181906 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:46.182001 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:46.182229 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 03:07:46.182352 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":105,"threads_started":1}
I20260812 03:07:46.184098 10025 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:07:46.184705 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.184440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:33307 (local address 127.0.0.1:38414)
0812 03:07:46.184546 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:46.184607 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:33307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 03:07:46.184892 9947 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.96.188:33307: connect: Connection refused (error 111)
I20260812 03:07:46.185602 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.185414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:33307 (local address 127.0.0.1:38420)
0812 03:07:46.185516 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:46.185556 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:33307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 03:07:46.185720 10025 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.96.188:33307: Network error: Client connection negotiation failed: client connection to 127.5.96.188:33307: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.188.keytab.
Entry for principal kudu/127.5.96.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.188.keytab.
I20260812 03:07:46.188091 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.188:33307
--webserver_interface=127.5.96.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.188.keytab
--principal=kudu/127.5.96.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:34271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:39919,127.5.96.189:36101,127.5.96.188:33307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:07:46.194267 9910 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.96.189" port: 36101 } attempt: 3
Aug 12 03:07:46 dist-test-slave-6bbx krb5kdc[9800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504065, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for kudu/127.5.96.189@KRBTEST.COM
I20260812 03:07:46.202106 9920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.194675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:36101 (local address 127.0.0.1:46686)
0812 03:07:46.194787 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:46.194800 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:46.194871 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:46.195071 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:46.195073 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:46.195210 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:46.195317 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:46.195322 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.196331 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.196333 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:46.196852 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:46.196859 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.197421 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.197423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:46.197502 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:46.197902 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:46.197913 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:46.199178 (+ 1265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:46.200861 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:46.200864 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:46.200865 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:46.201041 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:46.201276 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:46.201279 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:46.201280 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:46.201387 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:46.201739 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:46.201741 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:46.201816 (+ 75us) client_negotiation.cc:769] Sending connection context
0812 03:07:46.201887 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 03:07:46.201982 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 03:07:46.202203 10031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.194747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46686 (local address 127.5.96.189:36101)
0812 03:07:46.194856 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:46.194861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:46.194876 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:07:46.194934 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:46.194937 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:46.194981 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:46.195065 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:46.195419 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.196227 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.196955 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.197312 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.197420 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:46.199312 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:46.199316 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:46.199319 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:46.199343 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:46.200761 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.201119 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.201122 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.201123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.201200 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.201456 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.201459 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.201461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.201649 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:46.201730 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:46.201947 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 03:07:46.202050 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 03:07:46.203109 9910 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:07:46.203770 9920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.203509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:33307 (local address 127.0.0.1:38434)
0812 03:07:46.203615 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:46.203703 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:33307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 03:07:46.204376 9920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.204150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:33307 (local address 127.0.0.1:38448)
0812 03:07:46.204286 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:46.204328 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:33307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260812 03:07:46.204509 9910 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.96.188:33307: Network error: Client connection negotiation failed: client connection to 127.5.96.188:33307: connect: Connection refused (error 111)
I20260812 03:07:46.234929 10025 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 } attempt: 1
I20260812 03:07:46.235574 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.235334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:33307 (local address 127.0.0.1:38450)
0812 03:07:46.235453 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:46.235505 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:33307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 03:07:46.235651 9910 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 } attempt: 1
I20260812 03:07:46.236272 9920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.236038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:33307 (local address 127.0.0.1:38452)
0812 03:07:46.236160 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:46.236210 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:33307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 03:07:46.236703 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.236504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:33307 (local address 127.0.0.1:38462)
0812 03:07:46.236605 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:46.236649 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:33307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 03:07:46.236843 10025 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.96.188:33307: Network error: Client connection negotiation failed: client connection to 127.5.96.188:33307: connect: Connection refused (error 111)
I20260812 03:07:46.236925 9920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.236697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:33307 (local address 127.0.0.1:38466)
0812 03:07:46.236819 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:46.236868 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:33307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 03:07:46.237044 9910 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.96.188:33307: Network error: Client connection negotiation failed: client connection to 127.5.96.188:33307: connect: Connection refused (error 111)
W20260812 03:07:46.297108 10037 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:07:46.297353 10037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:46.297377 10037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:46.297432 10037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:46.301270 10037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:46.301313 10037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:46.301332 10037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:46.301362 10037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:46.301388 10037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:46.305054 10025 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 } attempt: 2
I20260812 03:07:46.305819 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.305525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:33307 (local address 127.0.0.1:38472)
0812 03:07:46.305670 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:46.305734 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:33307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 03:07:46.306442 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.306240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:33307 (local address 127.0.0.1:38480)
0812 03:07:46.306331 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:46.306379 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:33307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 03:07:46.306546 10025 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.96.188:33307: Network error: Client connection negotiation failed: client connection to 127.5.96.188:33307: connect: Connection refused (error 111)
I20260812 03:07:46.306720 10037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:34271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:39919,127.5.96.189:36101,127.5.96.188:33307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.188.keytab
--principal=kudu/127.5.96.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.188:33307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.96.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:46.307600 10037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:46.308534 10037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:07:46.312208 9910 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 } attempt: 2
I20260812 03:07:46.312919 9920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.312621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:33307 (local address 127.0.0.1:38492)
0812 03:07:46.312779 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:46.312839 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:33307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 03:07:46.313565 9920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.313359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:33307 (local address 127.0.0.1:38506)
0812 03:07:46.313452 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:46.313500 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:33307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 03:07:46.313681 9910 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.96.188:33307: Network error: Client connection negotiation failed: client connection to 127.5.96.188:33307: connect: Connection refused (error 111)
W20260812 03:07:46.316469 10050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:46.316478 10048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:46.316736 10037 server_base.cc:1034] running on GCE node
W20260812 03:07:46.316478 10043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:46.317260 10037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:46.318248 10037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:46.319401 10037 hybrid_clock.cc:648] HybridClock initialized: now 1786504066319386 us; error 24 us; skew 500 ppm
Aug 12 03:07:46 dist-test-slave-6bbx krb5kdc[9800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504066, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:46.321945 10037 init.cc:377] Logged in from keytab as kudu/127.5.96.188@KRBTEST.COM (short username kudu)
I20260812 03:07:46.322757 10037 webserver.cc:460] Webserver started at http://127.5.96.188:38299/ using document root <none> and password file <none>
I20260812 03:07:46.323256 10037 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:46.323323 10037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:46.323496 10037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:46.325474 10037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-2/data/instance:
uuid: "77c4086007424d5aa4d970808c63f4e0"
format_stamp: "Formatted at 2026-08-12 03:07:46 on dist-test-slave-6bbx"
server_key: "61898674c52ee153fb2419acea36f71a"
server_key_iv: "81781c78aa4875ed04c6ae3f715078fc"
server_key_version: "encryptionkey@0"
I20260812 03:07:46.326004 10037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-2/wal/instance:
uuid: "77c4086007424d5aa4d970808c63f4e0"
format_stamp: "Formatted at 2026-08-12 03:07:46 on dist-test-slave-6bbx"
server_key: "61898674c52ee153fb2419acea36f71a"
server_key_iv: "81781c78aa4875ed04c6ae3f715078fc"
server_key_version: "encryptionkey@0"
I20260812 03:07:46.329385 10037 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 03:07:46.331489 10063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:46.332608 10037 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:07:46.332718 10037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-2/wal
uuid: "77c4086007424d5aa4d970808c63f4e0"
format_stamp: "Formatted at 2026-08-12 03:07:46 on dist-test-slave-6bbx"
server_key: "61898674c52ee153fb2419acea36f71a"
server_key_iv: "81781c78aa4875ed04c6ae3f715078fc"
server_key_version: "encryptionkey@0"
I20260812 03:07:46.332811 10037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:46.366469 10037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:46.369239 10037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:46.369408 10037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:46.375854 10037 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.188:33307
I20260812 03:07:46.376091 10141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.188:33307 every 8 connection(s)
I20260812 03:07:46.376714 10037 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-2/data/info.pb
I20260812 03:07:46.379333 10143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:46.382905 10143 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39919 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:07:46.383749 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 10037
I20260812 03:07:46.383888 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786503790637001-5506-0/minicluster-data/master-2/wal/instance
I20260812 03:07:46.384146 5506 external_mini_cluster.cc:1428] Setting key 4ba3ac5eef04cb79d10e3386c01cdd30
Aug 12 03:07:46 dist-test-slave-6bbx krb5kdc[9800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504066, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.188@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
Aug 12 03:07:46 dist-test-slave-6bbx krb5kdc[9800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.188@KRBTEST.COM
I20260812 03:07:46.395407 9919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.384631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39334 (local address 127.5.96.190:39919)
0812 03:07:46.384747 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:46.384752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:46.385649 (+ 897us) server_negotiation.cc:407] Connection header received
0812 03:07:46.386383 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:46.386386 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:46.386435 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:46.386520 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:46.387914 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.388628 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.389593 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.389811 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.389897 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:46.391963 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:46.391966 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:46.391969 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:46.391995 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:46.393369 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.393895 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.393897 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.393899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.393963 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.394295 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.394299 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.394301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.394510 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:46.394596 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:46.395120 (+ 524us) server_negotiation.cc:299] Negotiation successful
0812 03:07:46.395236 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 03:07:46.396297 10147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.385070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38512 (local address 127.5.96.188:33307)
0812 03:07:46.385482 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:46.385496 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:46.385527 (+ 31us) server_negotiation.cc:407] Connection header received
0812 03:07:46.386178 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:46.386196 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:46.386402 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:46.386565 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:46.387237 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.388428 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.389219 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.389643 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.389802 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:46.392366 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:46.392373 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:46.392386 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:46.392411 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:46.394032 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.394430 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.394434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.394435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.394502 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.394746 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.394749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.394750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.394989 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:46.395149 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:46.395353 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 03:07:46.395467 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":302,"spinlock_wait_cycles":3328,"thread_start_us":161,"threads_started":1}
I20260812 03:07:46.396297 10146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.385069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:39919 (local address 127.0.0.1:39334)
0812 03:07:46.385505 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:46.385549 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:46.386190 (+ 641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:46.387298 (+ 1108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:46.387303 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:46.387595 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:46.387785 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:46.387796 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.388738 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.388741 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:46.389483 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:46.389492 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.389938 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.389942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:46.390032 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:46.390422 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:46.390436 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:46.391834 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:46.393470 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:46.393477 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:46.393486 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:46.393815 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:46.394034 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:46.394037 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:46.394039 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:46.394200 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:46.394605 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:46.394611 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:46.394866 (+ 255us) client_negotiation.cc:769] Sending connection context
0812 03:07:46.395062 (+ 196us) client_negotiation.cc:240] Negotiation successful
0812 03:07:46.395339 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":30,"thread_start_us":100,"threads_started":1}
I20260812 03:07:46.397660 10143 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.189" port: 36101 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:07:46.397687 10025 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 } attempt: 3
Aug 12 03:07:46 dist-test-slave-6bbx krb5kdc[9800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504066, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.189@KRBTEST.COM for kudu/127.5.96.188@KRBTEST.COM
Aug 12 03:07:46 dist-test-slave-6bbx krb5kdc[9800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504066, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.188@KRBTEST.COM for kudu/127.5.96.189@KRBTEST.COM
Aug 12 03:07:46 dist-test-slave-6bbx krb5kdc[9800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:07:46.406437 10147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.398124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38518 (local address 127.5.96.188:33307)
0812 03:07:46.398217 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:46.398231 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:46.398244 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:07:46.398390 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:46.398393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:46.398433 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:46.398925 (+ 492us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:46.399003 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.399715 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.400660 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.400995 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.401124 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:46.403074 (+ 1950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:46.403078 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:46.403081 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:46.403107 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:46.404567 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.405066 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.405069 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.405071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.405147 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.405446 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.405449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.405450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.405645 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:46.405745 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:46.406204 (+ 459us) server_negotiation.cc:299] Negotiation successful
0812 03:07:46.406296 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 03:07:46.406445 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.398065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:33307 (local address 127.0.0.1:38518)
0812 03:07:46.398183 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:46.398196 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:46.398284 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:46.398556 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:46.398560 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:46.398711 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:46.398811 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:46.398819 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.399817 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.399821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:46.400547 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:46.400555 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.401127 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.401131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:46.401209 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:46.401626 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:46.401641 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:46.402949 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:46.404688 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:46.404692 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:46.404694 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:46.404942 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:46.405256 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:46.405260 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:46.405262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:46.405342 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:46.405740 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:46.405743 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:46.405996 (+ 253us) client_negotiation.cc:769] Sending connection context
0812 03:07:46.406138 (+ 142us) client_negotiation.cc:240] Negotiation successful
0812 03:07:46.406261 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"spinlock_wait_cycles":333184}
I20260812 03:07:46.407114 10031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.398578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46696 (local address 127.5.96.189:36101)
0812 03:07:46.398683 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:46.398687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:46.398945 (+ 258us) server_negotiation.cc:407] Connection header received
0812 03:07:46.399106 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:46.399109 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:46.399148 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:46.399210 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:46.399559 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.400264 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.401058 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.401334 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.401660 (+ 326us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:46.403751 (+ 2091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:46.403755 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:46.403757 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:46.403783 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:46.405230 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.405653 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.405657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.405658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.405959 (+ 301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.406273 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.406276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.406278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.406494 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:46.406569 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:46.406890 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 03:07:46.406985 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:07:46.407505 10146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.398509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:36101 (local address 127.0.0.1:46696)
0812 03:07:46.398879 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:46.398891 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:46.399004 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:46.399234 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:46.399236 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:46.399380 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:46.399459 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:46.399463 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.400383 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.400386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:46.400961 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:46.400969 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.401673 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.401677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:46.401781 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:46.402205 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:46.402219 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:46.403557 (+ 1338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:46.405334 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:46.405337 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:46.405339 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:46.405563 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:46.406053 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:46.406056 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:46.406058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:46.406144 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:46.406645 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:46.406648 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:46.406726 (+ 78us) client_negotiation.cc:769] Sending connection context
0812 03:07:46.407241 (+ 515us) client_negotiation.cc:240] Negotiation successful
0812 03:07:46.407367 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312}
I20260812 03:07:46.408154 10143 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:07:46.408768 9919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.398846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39336 (local address 127.5.96.190:39919)
0812 03:07:46.398959 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:46.398962 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:46.399034 (+ 72us) server_negotiation.cc:407] Connection header received
0812 03:07:46.399170 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:46.399173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:46.399213 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:46.399273 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:46.399995 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.400442 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.401384 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.401595 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.401674 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:46.405058 (+ 3384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:46.405061 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:46.405064 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:46.405088 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:46.406980 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.407411 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.407414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.407416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.407508 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.407977 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.407980 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.407982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.408203 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:46.408408 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:46.408529 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 03:07:46.408629 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:07:46.409838 10025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24d7b7a09067448fb0994cd0f2f6231f: Bootstrap starting.
I20260812 03:07:46.412581 10025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24d7b7a09067448fb0994cd0f2f6231f: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:46.412823 9910 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 } attempt: 3
Aug 12 03:07:46 dist-test-slave-6bbx krb5kdc[9800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504066, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.188@KRBTEST.COM for kudu/127.5.96.188@KRBTEST.COM
I20260812 03:07:46.413935 10025 log.cc:824] T 00000000000000000000000000000000 P 24d7b7a09067448fb0994cd0f2f6231f: Log is configured to *not* fsync() on all Append() calls
Aug 12 03:07:46 dist-test-slave-6bbx krb5kdc[9800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.189@KRBTEST.COM
I20260812 03:07:46.416874 10025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24d7b7a09067448fb0994cd0f2f6231f: No bootstrap required, opened a new log
I20260812 03:07:46.417272 10147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.408696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38522 (local address 127.5.96.188:33307)
0812 03:07:46.409175 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:46.409178 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:46.409191 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:07:46.409220 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:46.409222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:46.409258 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:46.409336 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:46.409766 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.410411 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.411354 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.411629 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.411758 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:46.413867 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:46.413870 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:46.413873 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:46.413896 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:46.415478 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.415946 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.415949 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.415951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.416016 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.416234 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.416237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.416238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.416427 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:46.416501 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:46.417048 (+ 547us) server_negotiation.cc:299] Negotiation successful
0812 03:07:46.417137 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":416}
I20260812 03:07:46.417608 10146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.408520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:33307 (local address 127.0.0.1:38522)
0812 03:07:46.408765 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:46.408777 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:46.409015 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:46.409365 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:46.409368 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:46.409513 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:46.409591 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:46.409597 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.410532 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.410535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:46.411235 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:46.411246 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.411763 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.411766 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:46.411884 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:46.412318 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:46.412332 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:46.413708 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:46.415588 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:46.415591 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:46.415593 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:46.415854 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:46.416090 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:46.416092 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:46.416094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:46.416154 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:46.416529 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:46.416532 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:46.416604 (+ 72us) client_negotiation.cc:769] Sending connection context
0812 03:07:46.416687 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 03:07:46.416798 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":18}
I20260812 03:07:46.418579 10031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.409774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46704 (local address 127.5.96.189:36101)
0812 03:07:46.409872 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:46.409875 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:46.409948 (+ 73us) server_negotiation.cc:407] Connection header received
0812 03:07:46.410094 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:46.410097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:46.410135 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:46.410209 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:46.410796 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.411240 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.412102 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.412400 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.412503 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:46.415114 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:46.415118 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:46.415120 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:46.415142 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:46.416579 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.417042 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.417045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.417047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.417124 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.417413 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.417416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.417417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.417597 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:46.417770 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:46.418348 (+ 578us) server_negotiation.cc:299] Negotiation successful
0812 03:07:46.418447 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:07:46.420387 10143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77c4086007424d5aa4d970808c63f4e0: Bootstrap starting.
Aug 12 03:07:46 dist-test-slave-6bbx krb5kdc[9800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504065, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for kudu/127.5.96.188@KRBTEST.COM
I20260812 03:07:46.421286 10025 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24d7b7a09067448fb0994cd0f2f6231f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f2939d0c5348028f0b1d1502602793" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39919 } } peers { permanent_uuid: "24d7b7a09067448fb0994cd0f2f6231f" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 36101 } } peers { permanent_uuid: "77c4086007424d5aa4d970808c63f4e0" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 } }
I20260812 03:07:46.421499 10025 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24d7b7a09067448fb0994cd0f2f6231f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d7b7a09067448fb0994cd0f2f6231f, State: Initialized, Role: FOLLOWER
I20260812 03:07:46.421617 5506 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:07:46.421828 10025 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24d7b7a09067448fb0994cd0f2f6231f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f2939d0c5348028f0b1d1502602793" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39919 } } peers { permanent_uuid: "24d7b7a09067448fb0994cd0f2f6231f" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 36101 } } peers { permanent_uuid: "77c4086007424d5aa4d970808c63f4e0" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 } }
I20260812 03:07:46.422928 10143 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77c4086007424d5aa4d970808c63f4e0: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:46.423195 10025 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24d7b7a09067448fb0994cd0f2f6231f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:46.423806 10143 log.cc:824] T 00000000000000000000000000000000 P 77c4086007424d5aa4d970808c63f4e0: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:46.423014 9875 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d0f2939d0c5348028f0b1d1502602793"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39336
I20260812 03:07:46.424796 10151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24d7b7a09067448fb0994cd0f2f6231f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f2939d0c5348028f0b1d1502602793" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39919 } } peers { permanent_uuid: "24d7b7a09067448fb0994cd0f2f6231f" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 36101 } } peers { permanent_uuid: "77c4086007424d5aa4d970808c63f4e0" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 } } }
I20260812 03:07:46.424993 9920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.413152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:33307 (local address 127.0.0.1:38530)
0812 03:07:46.413303 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:46.413318 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:46.413388 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:46.414732 (+ 1344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:46.414735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:46.414895 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:46.414988 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:46.414995 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.416466 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.416469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:46.417204 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:46.417214 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.417825 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.417829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:46.417907 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:46.419756 (+ 1849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:46.419770 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:46.421104 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:46.423400 (+ 2296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:46.423403 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:46.423405 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:46.423639 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:46.424069 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:46.424072 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:46.424074 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:46.424169 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:46.424526 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:46.424529 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:46.424620 (+ 91us) client_negotiation.cc:769] Sending connection context
0812 03:07:46.424722 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 03:07:46.424846 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260812 03:07:46.425009 10151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24d7b7a09067448fb0994cd0f2f6231f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:07:46.425668 10148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.413188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38530 (local address 127.5.96.188:33307)
0812 03:07:46.414492 (+ 1304us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:46.414502 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:46.414514 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:07:46.414545 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:46.414547 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:46.414584 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:46.415101 (+ 517us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:46.415198 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.416346 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.417345 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.417686 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.418333 (+ 647us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:46.421246 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:46.421250 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:46.421252 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:46.421275 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:46.423234 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.423789 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.423792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.423793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.423957 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.424280 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.424282 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.424283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.424443 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:46.424530 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:46.425411 (+ 881us) server_negotiation.cc:299] Negotiation successful
0812 03:07:46.425520 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1241,"thread_start_us":128,"threads_started":1}
I20260812 03:07:46.426838 10143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77c4086007424d5aa4d970808c63f4e0: No bootstrap required, opened a new log
I20260812 03:07:46.428354 9910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793: Bootstrap starting.
W20260812 03:07:46.428453 10174 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 24d7b7a09067448fb0994cd0f2f6231f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:07:46.428514 10174 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:07:46.426759 9927 master_runner.cc:186] Error getting master registration for 127.5.96.190:39919: OK, instance_id { permanent_uuid: "d0f2939d0c5348028f0b1d1502602793" instance_seqno: 1786504065954972 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:07:46.429914 9927 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:07:46.430588 10143 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77c4086007424d5aa4d970808c63f4e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f2939d0c5348028f0b1d1502602793" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39919 } } peers { permanent_uuid: "24d7b7a09067448fb0994cd0f2f6231f" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 36101 } } peers { permanent_uuid: "77c4086007424d5aa4d970808c63f4e0" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 } }
I20260812 03:07:46.430765 9910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:46.430826 10143 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77c4086007424d5aa4d970808c63f4e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77c4086007424d5aa4d970808c63f4e0, State: Initialized, Role: FOLLOWER
I20260812 03:07:46.431150 10143 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77c4086007424d5aa4d970808c63f4e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f2939d0c5348028f0b1d1502602793" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39919 } } peers { permanent_uuid: "24d7b7a09067448fb0994cd0f2f6231f" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 36101 } } peers { permanent_uuid: "77c4086007424d5aa4d970808c63f4e0" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 } }
I20260812 03:07:46.431568 9910 log.cc:824] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:46.431728 10182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77c4086007424d5aa4d970808c63f4e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f2939d0c5348028f0b1d1502602793" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39919 } } peers { permanent_uuid: "24d7b7a09067448fb0994cd0f2f6231f" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 36101 } } peers { permanent_uuid: "77c4086007424d5aa4d970808c63f4e0" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 } } }
I20260812 03:07:46.431948 10182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77c4086007424d5aa4d970808c63f4e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:07:46.432396 10143 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77c4086007424d5aa4d970808c63f4e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:46.437431 9910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793: No bootstrap required, opened a new log
I20260812 03:07:46.436036 10037 master_runner.cc:186] Error getting master registration for 127.5.96.190:39919: OK, instance_id { permanent_uuid: "d0f2939d0c5348028f0b1d1502602793" instance_seqno: 1786504065954972 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:07:46.439086 10037 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:07:46.440279 9910 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f2939d0c5348028f0b1d1502602793" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39919 } } peers { permanent_uuid: "24d7b7a09067448fb0994cd0f2f6231f" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 36101 } } peers { permanent_uuid: "77c4086007424d5aa4d970808c63f4e0" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 } }
I20260812 03:07:46.440438 9910 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f2939d0c5348028f0b1d1502602793, State: Initialized, Role: FOLLOWER
I20260812 03:07:46.440663 9910 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f2939d0c5348028f0b1d1502602793" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39919 } } peers { permanent_uuid: "24d7b7a09067448fb0994cd0f2f6231f" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 36101 } } peers { permanent_uuid: "77c4086007424d5aa4d970808c63f4e0" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 } }
I20260812 03:07:46.441046 10202 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f2939d0c5348028f0b1d1502602793" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39919 } } peers { permanent_uuid: "24d7b7a09067448fb0994cd0f2f6231f" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 36101 } } peers { permanent_uuid: "77c4086007424d5aa4d970808c63f4e0" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 } } }
I20260812 03:07:46.441174 10202 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:07:46.441717 9910 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:46.441830 9875 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 03:07:46.441944 9875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:46.443150 9875 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f2939d0c5348028f0b1d1502602793" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39919 } } peers { permanent_uuid: "24d7b7a09067448fb0994cd0f2f6231f" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 36101 } } peers { permanent_uuid: "77c4086007424d5aa4d970808c63f4e0" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 } }
I20260812 03:07:46.444538 9981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d0f2939d0c5348028f0b1d1502602793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "24d7b7a09067448fb0994cd0f2f6231f"
I20260812 03:07:46.444741 9981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24d7b7a09067448fb0994cd0f2f6231f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:46.445950 9981 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 24d7b7a09067448fb0994cd0f2f6231f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0f2939d0c5348028f0b1d1502602793 in term 1.
I20260812 03:07:46.446472 9836 leader_election.cc:302] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24d7b7a09067448fb0994cd0f2f6231f, d0f2939d0c5348028f0b1d1502602793; no voters:
I20260812 03:07:46.446732 10202 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793 [term 1 FOLLOWER]: Leader election won for term 1
W20260812 03:07:46.446877 10201 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 77c4086007424d5aa4d970808c63f4e0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:07:46.446942 10201 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:07:46.446923 10202 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793 [term 1 LEADER]: Becoming Leader. State: Replica: d0f2939d0c5348028f0b1d1502602793, State: Running, Role: LEADER
I20260812 03:07:46.447216 10202 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f2939d0c5348028f0b1d1502602793" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39919 } } peers { permanent_uuid: "24d7b7a09067448fb0994cd0f2f6231f" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 36101 } } peers { permanent_uuid: "77c4086007424d5aa4d970808c63f4e0" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 } }
I20260812 03:07:46.448534 9875 leader_election.cc:288] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793 [CANDIDATE]: Term 1 election: Requested vote from peers 24d7b7a09067448fb0994cd0f2f6231f (127.5.96.189:36101), 77c4086007424d5aa4d970808c63f4e0 (127.5.96.188:33307)
I20260812 03:07:46.448961 10202 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0f2939d0c5348028f0b1d1502602793. Latest consensus state: current_term: 1 leader_uuid: "d0f2939d0c5348028f0b1d1502602793" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f2939d0c5348028f0b1d1502602793" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39919 } } peers { permanent_uuid: "24d7b7a09067448fb0994cd0f2f6231f" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 36101 } } peers { permanent_uuid: "77c4086007424d5aa4d970808c63f4e0" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 } } }
I20260812 03:07:46.449067 10202 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:46.449859 10103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d0f2939d0c5348028f0b1d1502602793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "77c4086007424d5aa4d970808c63f4e0"
I20260812 03:07:46.450067 10103 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77c4086007424d5aa4d970808c63f4e0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:46.451263 10103 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 77c4086007424d5aa4d970808c63f4e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0f2939d0c5348028f0b1d1502602793 in term 1.
I20260812 03:07:46.452765 9814 master_runner.cc:263] No leader master found from master d0f2939d0c5348028f0b1d1502602793
I20260812 03:07:46.452828 9814 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 03:07:46.452914 9843 tablet.cc:2369] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:07:46.455085 10217 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:07:46.455166 10217 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:07:46.455225 10226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:46.456188 10226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:46.460160 10031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.450467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46720 (local address 127.5.96.189:36101)
0812 03:07:46.450571 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:46.450575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:46.451038 (+ 463us) server_negotiation.cc:407] Connection header received
0812 03:07:46.451205 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:46.451208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:46.451264 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:46.451313 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:46.452414 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.453171 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.454398 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.454729 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.454795 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:46.456776 (+ 1981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:46.456780 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:46.456783 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:46.456807 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:46.458372 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.458844 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.458848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.458850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.458931 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.459213 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.459216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.459218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.459429 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:46.459526 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:46.459894 (+ 368us) server_negotiation.cc:299] Negotiation successful
0812 03:07:46.460010 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:07:46.469645 10102 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 77c4086007424d5aa4d970808c63f4e0 [term 1 FOLLOWER]: Refusing update from remote peer d0f2939d0c5348028f0b1d1502602793: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:07:46.470458 9981 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 24d7b7a09067448fb0994cd0f2f6231f [term 1 FOLLOWER]: Refusing update from remote peer d0f2939d0c5348028f0b1d1502602793: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:07:46.469691 10148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.455927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38542 (local address 127.5.96.188:33307)
0812 03:07:46.456076 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:46.456081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:46.460822 (+ 4741us) server_negotiation.cc:407] Connection header received
0812 03:07:46.461004 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:46.461007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:46.461052 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:46.461101 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:46.461914 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.462698 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.463859 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.464578 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.464651 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:46.466205 (+ 1554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:46.466209 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:46.466213 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:46.466239 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:46.467832 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.468406 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.468409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.468411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.468498 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.468782 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.468786 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.468788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.468988 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:46.469082 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:46.469424 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 03:07:46.469537 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 03:07:46.471084 10202 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77c4086007424d5aa4d970808c63f4e0" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:07:46.471979 10202 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d7b7a09067448fb0994cd0f2f6231f" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 36101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:07:46.477149 10182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77c4086007424d5aa4d970808c63f4e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0f2939d0c5348028f0b1d1502602793. Latest consensus state: current_term: 1 leader_uuid: "d0f2939d0c5348028f0b1d1502602793" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f2939d0c5348028f0b1d1502602793" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39919 } } peers { permanent_uuid: "24d7b7a09067448fb0994cd0f2f6231f" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 36101 } } peers { permanent_uuid: "77c4086007424d5aa4d970808c63f4e0" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 } } }
I20260812 03:07:46.477264 10182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77c4086007424d5aa4d970808c63f4e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:07:46.479805 10151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24d7b7a09067448fb0994cd0f2f6231f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0f2939d0c5348028f0b1d1502602793. Latest consensus state: current_term: 1 leader_uuid: "d0f2939d0c5348028f0b1d1502602793" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f2939d0c5348028f0b1d1502602793" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39919 } } peers { permanent_uuid: "24d7b7a09067448fb0994cd0f2f6231f" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 36101 } } peers { permanent_uuid: "77c4086007424d5aa4d970808c63f4e0" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 } } }
I20260812 03:07:46.479950 10151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24d7b7a09067448fb0994cd0f2f6231f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:07:46.483884 9919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:46.460551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39344 (local address 127.5.96.190:39919)
0812 03:07:46.460651 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:46.460655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:46.461876 (+ 1221us) server_negotiation.cc:407] Connection header received
0812 03:07:46.462051 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:46.462054 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:46.462113 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:46.462164 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:46.471245 (+ 9081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.472036 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.473140 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:46.473487 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:46.473560 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:46.480476 (+ 6916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:46.480480 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:46.480483 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:46.480508 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:46.482084 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.482585 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.482589 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.482591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.482672 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:46.482951 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:46.482955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:46.482956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:46.483153 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:46.483259 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:46.483603 (+ 344us) server_negotiation.cc:299] Negotiation successful
0812 03:07:46.483743 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:07:46.488303 10222 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0f2939d0c5348028f0b1d1502602793" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f2939d0c5348028f0b1d1502602793" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39919 } } peers { permanent_uuid: "24d7b7a09067448fb0994cd0f2f6231f" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 36101 } } peers { permanent_uuid: "77c4086007424d5aa4d970808c63f4e0" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 } } }
I20260812 03:07:46.488420 10222 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:46.489116 10182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77c4086007424d5aa4d970808c63f4e0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0f2939d0c5348028f0b1d1502602793" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f2939d0c5348028f0b1d1502602793" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39919 } } peers { permanent_uuid: "24d7b7a09067448fb0994cd0f2f6231f" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 36101 } } peers { permanent_uuid: "77c4086007424d5aa4d970808c63f4e0" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 } } }
I20260812 03:07:46.489225 10182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77c4086007424d5aa4d970808c63f4e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:07:46.489475 10238 mvcc.cc:205] Tried to move back new op lower bound from 7317520656250052608 to 7317520656171200512. Current Snapshot: MvccSnapshot[applied={T|T < 7317520656250052608}]
I20260812 03:07:46.490063 10226 catalog_manager.cc:1347] Generated new cluster ID: abc8592580a847ac9359170375a2fb81
I20260812 03:07:46.490129 10226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:46.499312 10202 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0f2939d0c5348028f0b1d1502602793" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f2939d0c5348028f0b1d1502602793" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39919 } } peers { permanent_uuid: "24d7b7a09067448fb0994cd0f2f6231f" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 36101 } } peers { permanent_uuid: "77c4086007424d5aa4d970808c63f4e0" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 } } }
I20260812 03:07:46.499418 10202 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:46.502065 10151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24d7b7a09067448fb0994cd0f2f6231f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0f2939d0c5348028f0b1d1502602793" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f2939d0c5348028f0b1d1502602793" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 39919 } } peers { permanent_uuid: "24d7b7a09067448fb0994cd0f2f6231f" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 36101 } } peers { permanent_uuid: "77c4086007424d5aa4d970808c63f4e0" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 33307 } } }
I20260812 03:07:46.502180 10151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24d7b7a09067448fb0994cd0f2f6231f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:07:46.502866 10240 mvcc.cc:205] Tried to move back new op lower bound from 7317520656250052608 to 7317520656171200512. Current Snapshot: MvccSnapshot[applied={T|T < 7317520656250052608}]
I20260812 03:07:46.517603 10226 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:46.518729 10226 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:46.531131 10226 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0f2939d0c5348028f0b1d1502602793: Generated new TSK 0
I20260812 03:07:46.531876 10226 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:07:46.574710 10148 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:07:46.572842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38554 (local address 127.5.96.188:33307)
0812 03:07:46.572958 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:46.572961 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:46.573662 (+ 701us) server_negotiation.cc:407] Connection header received
0812 03:07:46.573867 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:46.573870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:46.573918 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:46.573966 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:46.574572 (+ 606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38554: BlockingRecv error: recv got EOF from 127.0.0.1:38554 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260812 03:07:46.576128 10031 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:07:46.572392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46724 (local address 127.5.96.189:36101)
0812 03:07:46.572498 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:46.572502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:46.575223 (+ 2721us) server_negotiation.cc:407] Connection header received
0812 03:07:46.575399 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:46.575402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:46.575452 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:46.575502 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:46.576011 (+ 509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46724: BlockingRecv error: recv got EOF from 127.0.0.1:46724 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260812 03:07:46.577456 9919 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:07:46.572017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39348 (local address 127.5.96.190:39919)
0812 03:07:46.572144 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:46.572149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:46.576570 (+ 4421us) server_negotiation.cc:407] Connection header received
0812 03:07:46.576735 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:46.576739 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:46.576793 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:46.576846 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:46.577335 (+ 489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39348: BlockingRecv error: recv got EOF from 127.0.0.1:39348 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260812 03:07:46.581930 10031 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:07:46.580390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46734 (local address 127.5.96.189:36101)
0812 03:07:46.580486 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:46.580490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:46.581024 (+ 534us) server_negotiation.cc:407] Connection header received
0812 03:07:46.581223 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:46.581226 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:46.581297 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:46.581347 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:46.581837 (+ 490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46734: BlockingRecv error: recv got EOF from 127.0.0.1:46734 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260812 03:07:46.588867 10031 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:07:46.585318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46740 (local address 127.5.96.189:36101)
0812 03:07:46.585412 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:46.585416 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:46.588031 (+ 2615us) server_negotiation.cc:407] Connection header received
0812 03:07:46.588204 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:46.588207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:46.588255 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:46.588303 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:46.588773 (+ 470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46740: BlockingRecv error: recv got EOF from 127.0.0.1:46740 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260812 03:07:46.594415 10031 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:07:46.592940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46754 (local address 127.5.96.189:36101)
0812 03:07:46.593036 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:46.593039 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:46.593594 (+ 555us) server_negotiation.cc:407] Connection header received
0812 03:07:46.593780 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:46.593783 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:46.593829 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:46.593876 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:46.594326 (+ 450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46754: BlockingRecv error: recv got EOF from 127.0.0.1:46754 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260812 03:07:46.601948 10031 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:07:46.600420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46770 (local address 127.5.96.189:36101)
0812 03:07:46.600541 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:46.600546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:46.601119 (+ 573us) server_negotiation.cc:407] Connection header received
0812 03:07:46.601293 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:46.601296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:46.601343 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:46.601390 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:46.601857 (+ 467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46770: BlockingRecv error: recv got EOF from 127.0.0.1:46770 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260812 03:07:46.614009 10031 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:07:46.612264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46772 (local address 127.5.96.189:36101)
0812 03:07:46.612379 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:46.612383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:46.613033 (+ 650us) server_negotiation.cc:407] Connection header received
0812 03:07:46.613242 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:46.613246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:46.613300 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:46.613356 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:46.613909 (+ 553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46772: BlockingRecv error: recv got EOF from 127.0.0.1:46772 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260812 03:07:46.633999 10031 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:07:46.632376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46782 (local address 127.5.96.189:36101)
0812 03:07:46.632483 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:46.632487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:46.633007 (+ 520us) server_negotiation.cc:407] Connection header received
0812 03:07:46.633201 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:46.633205 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:46.633260 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:46.633311 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:46.633885 (+ 574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46782: BlockingRecv error: recv got EOF from 127.0.0.1:46782 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260812 03:07:46.669243 10031 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:07:46.668191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46788 (local address 127.5.96.189:36101)
0812 03:07:46.668318 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:46.668323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:46.668466 (+ 143us) server_negotiation.cc:407] Connection header received
0812 03:07:46.668595 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:46.668597 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:46.668650 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:46.668730 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:46.669129 (+ 399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46788: BlockingRecv error: recv got EOF from 127.0.0.1:46788 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260812 03:07:46.736230 10031 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:07:46.735145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46804 (local address 127.5.96.189:36101)
0812 03:07:46.735288 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:46.735293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:46.735425 (+ 132us) server_negotiation.cc:407] Connection header received
0812 03:07:46.735546 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:46.735549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:46.735598 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:46.735702 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:46.736101 (+ 399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46804: BlockingRecv error: recv got EOF from 127.0.0.1:46804 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260812 03:07:46.867460 10031 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:07:46.866278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46812 (local address 127.5.96.189:36101)
0812 03:07:46.866432 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:46.866437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:46.866578 (+ 141us) server_negotiation.cc:407] Connection header received
0812 03:07:46.866731 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:46.866734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:46.866794 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:46.866869 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:46.867339 (+ 470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46812: BlockingRecv error: recv got EOF from 127.0.0.1:46812 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260812 03:07:47.127099 10031 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:07:47.125886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46814 (local address 127.5.96.189:36101)
0812 03:07:47.126020 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:47.126025 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:47.126330 (+ 305us) server_negotiation.cc:407] Connection header received
0812 03:07:47.126480 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:47.126483 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:47.126538 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:47.126599 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:47.127000 (+ 401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46814: BlockingRecv error: recv got EOF from 127.0.0.1:46814 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:07:47.429833 10174 catalog_manager.cc:1259] Loaded cluster ID: abc8592580a847ac9359170375a2fb81
I20260812 03:07:47.429916 10174 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 24d7b7a09067448fb0994cd0f2f6231f: loading cluster ID for follower catalog manager: success
I20260812 03:07:47.431625 10174 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 24d7b7a09067448fb0994cd0f2f6231f: acquiring CA information for follower catalog manager: success
I20260812 03:07:47.432835 10174 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 24d7b7a09067448fb0994cd0f2f6231f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:07:47.448036 10201 catalog_manager.cc:1259] Loaded cluster ID: abc8592580a847ac9359170375a2fb81
I20260812 03:07:47.448093 10201 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 77c4086007424d5aa4d970808c63f4e0: loading cluster ID for follower catalog manager: success
I20260812 03:07:47.449555 10201 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 77c4086007424d5aa4d970808c63f4e0: acquiring CA information for follower catalog manager: success
I20260812 03:07:47.450543 10201 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 77c4086007424d5aa4d970808c63f4e0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:07:47.644942 10343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:47.640974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46816 (local address 127.5.96.189:36101)
0812 03:07:47.641207 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:47.641212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:47.641353 (+ 141us) server_negotiation.cc:407] Connection header received
0812 03:07:47.641509 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:47.641511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:47.641641 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:47.641736 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:07:47.642017 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:47.642671 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:47.643512 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:47.643793 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:47.643871 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:47.644127 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:07:47.644608 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:07:47.644703 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:47.644730 (+ 27us) server_negotiation.cc:299] Negotiation successful
0812 03:07:47.644782 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":75,"threads_started":1}
I20260812 03:07:47.650858 10351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:47.647809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38564 (local address 127.5.96.188:33307)
0812 03:07:47.648026 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:47.648029 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:47.648042 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:07:47.648119 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:47.648121 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:47.648236 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:47.648330 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:07:47.648471 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:47.649001 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:47.649721 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:47.649929 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:47.649995 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:47.650189 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:07:47.650556 (+ 367us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:07:47.650636 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:47.650701 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 03:07:47.650745 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":76,"threads_started":1}
I20260812 03:07:47.657298 10343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:47.653444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46828 (local address 127.5.96.189:36101)
0812 03:07:47.653648 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:47.653651 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:47.654041 (+ 390us) server_negotiation.cc:407] Connection header received
0812 03:07:47.654153 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:47.654155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:47.654196 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:47.654282 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:07:47.654485 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:47.655007 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:47.656038 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:47.656361 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:47.656457 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:47.656671 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:07:47.656748 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:07:47.656818 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:47.657128 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 03:07:47.657182 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260812 03:07:47.657356 10351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:47.653528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38576 (local address 127.5.96.188:33307)
0812 03:07:47.653642 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:47.653647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:47.654045 (+ 398us) server_negotiation.cc:407] Connection header received
0812 03:07:47.654216 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:47.654219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:47.654262 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:47.654325 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:07:47.654596 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:47.655335 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:47.656315 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:47.656582 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:47.656667 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:47.656943 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:07:47.657009 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:07:47.657079 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:47.657210 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 03:07:47.657261 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:07:47.657979 10362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:47.653452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39354 (local address 127.5.96.190:39919)
0812 03:07:47.653682 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:47.653686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:47.653890 (+ 204us) server_negotiation.cc:407] Connection header received
0812 03:07:47.654042 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:47.654045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:47.654195 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:47.654301 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:07:47.654645 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:47.655446 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:47.656486 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:47.656775 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:47.656868 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:47.657249 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:07:47.657544 (+ 295us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:07:47.657627 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:47.657769 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 03:07:47.657833 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":69,"threads_started":1}
I20260812 03:07:47.660756 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 9814
I20260812 03:07:47.680737 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 9927
I20260812 03:07:47.690585 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 10037
2026-08-12T03:07:47Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3173 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:07:47 dist-test-slave-6bbx krb5kdc[10370](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:07:47 dist-test-slave-6bbx krb5kdc[10370](info): set up 2 sockets
Aug 12 03:07:47 dist-test-slave-6bbx krb5kdc[10370](info): commencing operation
krb5kdc: starting...
W20260812 03:07:48.843207 5506 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/krb5kdc/test-user.keytab.
Aug 12 03:07:48 dist-test-slave-6bbx krb5kdc[10370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:07:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.96.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
Entry for principal kudu/127.5.96.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab.
I20260812 03:07:48.956072 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:35993
--webserver_interface=127.5.96.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:35503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:35993,127.5.96.189:37281,127.5.96.188:39011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:49.061651 10392 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:07:49.061901 10392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:49.061934 10392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:49.061992 10392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:49.065846 10392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:49.065893 10392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:49.065919 10392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:49.065940 10392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:49.065968 10392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:49.070847 10392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:35503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:35993,127.5.96.189:37281,127.5.96.188:39011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.190.keytab
--principal=kudu/127.5.96.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.190:35993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.96.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:49.071774 10392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:49.072679 10392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:49.081796 10404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:49.081804 10408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:49.081804 10402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:49.081959 10392 server_base.cc:1034] running on GCE node
I20260812 03:07:49.082664 10392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:49.083634 10392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:49.084816 10392 hybrid_clock.cc:648] HybridClock initialized: now 1786504069084795 us; error 32 us; skew 500 ppm
Aug 12 03:07:49 dist-test-slave-6bbx krb5kdc[10370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504069, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:49.087395 10392 init.cc:377] Logged in from keytab as kudu/127.5.96.190@KRBTEST.COM (short username kudu)
I20260812 03:07:49.088217 10392 webserver.cc:460] Webserver started at http://127.5.96.190:44089/ using document root <none> and password file <none>
I20260812 03:07:49.088704 10392 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:49.088765 10392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:49.088954 10392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:49.090852 10392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-0/data/instance:
uuid: "2fb47476959a44518470346eec9b49e4"
format_stamp: "Formatted at 2026-08-12 03:07:49 on dist-test-slave-6bbx"
server_key: "d4e75abb7c1eb991e4c72b29ff66362d"
server_key_iv: "d13bbedb33bac15f752b6a8fa73b0c8c"
server_key_version: "encryptionkey@0"
I20260812 03:07:49.091369 10392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-0/wal/instance:
uuid: "2fb47476959a44518470346eec9b49e4"
format_stamp: "Formatted at 2026-08-12 03:07:49 on dist-test-slave-6bbx"
server_key: "d4e75abb7c1eb991e4c72b29ff66362d"
server_key_iv: "d13bbedb33bac15f752b6a8fa73b0c8c"
server_key_version: "encryptionkey@0"
I20260812 03:07:49.094799 10392 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 03:07:49.096976 10423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:49.098064 10392 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:07:49.098169 10392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-0/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-0/wal
uuid: "2fb47476959a44518470346eec9b49e4"
format_stamp: "Formatted at 2026-08-12 03:07:49 on dist-test-slave-6bbx"
server_key: "d4e75abb7c1eb991e4c72b29ff66362d"
server_key_iv: "d13bbedb33bac15f752b6a8fa73b0c8c"
server_key_version: "encryptionkey@0"
I20260812 03:07:49.098286 10392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:49.112161 10392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:49.114979 10392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:49.115139 10392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:49.121531 10392 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.190:35993
I20260812 03:07:49.121790 10501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.190:35993 every 8 connection(s)
I20260812 03:07:49.122402 10392 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:49.125033 10502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:49.128475 10502 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35993 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:07:49.131035 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 10392
I20260812 03:07:49.131146 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-0/wal/instance
I20260812 03:07:49.131400 5506 external_mini_cluster.cc:1428] Setting key fecd7091563493bbceed0103d54c1c07
Aug 12 03:07:49 dist-test-slave-6bbx krb5kdc[10370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504069, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.96.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.189@KRBTEST.COM" created.
I20260812 03:07:49.142488 10509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.130434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46550 (local address 127.5.96.190:35993)
0812 03:07:49.130860 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:49.130878 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:49.131010 (+ 132us) server_negotiation.cc:407] Connection header received
0812 03:07:49.132047 (+ 1037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:49.132053 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:49.132286 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:49.132468 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:49.133151 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.134109 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.135369 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.135763 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.135873 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:49.138249 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:49.138255 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:49.138263 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:49.138284 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:49.139703 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.140123 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.140128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.140130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.140250 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.140523 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.140526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.140527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.140862 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:49.140977 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:49.141316 (+ 339us) server_negotiation.cc:299] Negotiation successful
0812 03:07:49.141513 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":284,"spinlock_wait_cycles":16128,"thread_start_us":133,"threads_started":1}
I20260812 03:07:49.142488 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.130435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:35993 (local address 127.0.0.1:46550)
0812 03:07:49.130865 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:49.130911 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:49.131746 (+ 835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:49.132457 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:49.132462 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:49.132775 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:49.133016 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:49.133028 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.134233 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.134236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:49.135265 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:49.135276 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.135899 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.135903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:49.136078 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:49.136606 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:49.136627 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:49.138121 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:49.139800 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:49.139808 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:49.139818 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:49.140026 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:49.140333 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:49.140336 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:49.140338 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:49.140441 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:49.140968 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:49.140974 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:49.141099 (+ 125us) client_negotiation.cc:769] Sending connection context
0812 03:07:49.141246 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 03:07:49.141579 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":116,"thread_start_us":95,"threads_started":1}
I20260812 03:07:49.144614 10502 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.189" port: 37281 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:07:49.145254 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.144997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:37281 (local address 127.0.0.1:57212)
0812 03:07:49.145114 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:49.145188 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.189:37281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 03:07:49.145448 10426 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.96.189:37281: connect: Connection refused (error 111)
I20260812 03:07:49.146220 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.145980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:37281 (local address 127.0.0.1:57228)
0812 03:07:49.146101 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:49.146158 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.189:37281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 03:07:49.146371 10502 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.96.189:37281: Network error: Client connection negotiation failed: client connection to 127.5.96.189:37281: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.189.keytab.
Entry for principal kudu/127.5.96.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.189.keytab.
I20260812 03:07:49.155710 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.189:37281
--webserver_interface=127.5.96.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.189.keytab
--principal=kudu/127.5.96.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:35503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:35993,127.5.96.189:37281,127.5.96.188:39011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:07:49.195622 10502 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.96.189" port: 37281 } attempt: 1
I20260812 03:07:49.196432 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.196127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:37281 (local address 127.0.0.1:57236)
0812 03:07:49.196281 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:49.196349 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.189:37281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 03:07:49.197156 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.196931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:37281 (local address 127.0.0.1:57248)
0812 03:07:49.197038 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:49.197090 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.189:37281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 03:07:49.197309 10502 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.96.189:37281: Network error: Client connection negotiation failed: client connection to 127.5.96.189:37281: connect: Connection refused (error 111)
W20260812 03:07:49.261736 10515 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:07:49.261979 10515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:49.262022 10515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:49.262081 10515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 03:07:49.265468 10502 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.96.189" port: 37281 } attempt: 2
W20260812 03:07:49.266081 10515 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:49.266141 10515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:49.266165 10515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:49.266179 10515 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:49.266194 10515 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:49.266326 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.265985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:37281 (local address 127.0.0.1:57262)
0812 03:07:49.266167 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:49.266245 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.189:37281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260812 03:07:49.267019 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.266813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:37281 (local address 127.0.0.1:57276)
0812 03:07:49.266913 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:49.266959 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.189:37281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 03:07:49.267163 10502 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.96.189:37281: Network error: Client connection negotiation failed: client connection to 127.5.96.189:37281: connect: Connection refused (error 111)
I20260812 03:07:49.271371 10515 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:35503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:35993,127.5.96.189:37281,127.5.96.188:39011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.189.keytab
--principal=kudu/127.5.96.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.189:37281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.96.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:49.272336 10515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:49.273262 10515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:49.280982 10524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:49.280975 10521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:49.281087 10515 server_base.cc:1034] running on GCE node
W20260812 03:07:49.280982 10526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:49.281718 10515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:49.282694 10515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:49.283840 10515 hybrid_clock.cc:648] HybridClock initialized: now 1786504069283824 us; error 26 us; skew 500 ppm
Aug 12 03:07:49 dist-test-slave-6bbx krb5kdc[10370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504069, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:49.286296 10515 init.cc:377] Logged in from keytab as kudu/127.5.96.189@KRBTEST.COM (short username kudu)
I20260812 03:07:49.287111 10515 webserver.cc:460] Webserver started at http://127.5.96.189:38371/ using document root <none> and password file <none>
I20260812 03:07:49.287592 10515 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:49.287650 10515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:49.287881 10515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:49.289764 10515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-1/data/instance:
uuid: "72920dc6e86f4a26bdf93656ee298281"
format_stamp: "Formatted at 2026-08-12 03:07:49 on dist-test-slave-6bbx"
server_key: "c8d2a5a1e5d96f007d1afcb9a4c4858b"
server_key_iv: "89e97ca9f4266b794586969230089afb"
server_key_version: "encryptionkey@0"
I20260812 03:07:49.290271 10515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-1/wal/instance:
uuid: "72920dc6e86f4a26bdf93656ee298281"
format_stamp: "Formatted at 2026-08-12 03:07:49 on dist-test-slave-6bbx"
server_key: "c8d2a5a1e5d96f007d1afcb9a4c4858b"
server_key_iv: "89e97ca9f4266b794586969230089afb"
server_key_version: "encryptionkey@0"
I20260812 03:07:49.293700 10515 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 03:07:49.295855 10535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:49.296940 10515 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:07:49.297058 10515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-1/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-1/wal
uuid: "72920dc6e86f4a26bdf93656ee298281"
format_stamp: "Formatted at 2026-08-12 03:07:49 on dist-test-slave-6bbx"
server_key: "c8d2a5a1e5d96f007d1afcb9a4c4858b"
server_key_iv: "89e97ca9f4266b794586969230089afb"
server_key_version: "encryptionkey@0"
I20260812 03:07:49.297142 10515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:49.305698 10515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:49.308060 10515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:49.308213 10515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:49.314399 10515 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.189:37281
I20260812 03:07:49.314621 10604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.189:37281 every 8 connection(s)
I20260812 03:07:49.315231 10515 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-1/data/info.pb
I20260812 03:07:49.317749 10606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:49.320986 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 10515
I20260812 03:07:49.321110 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-1/wal/instance
I20260812 03:07:49.321411 5506 external_mini_cluster.cc:1428] Setting key e2f88f8bcff3452a5730d6938eeeafa1
I20260812 03:07:49.321411 10606 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35993 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.96.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.96.188@KRBTEST.COM" created.
Aug 12 03:07:49 dist-test-slave-6bbx krb5kdc[10370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504069, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.189@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:07:49.336395 10509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.323407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46552 (local address 127.5.96.190:35993)
0812 03:07:49.323531 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:49.323535 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:49.324405 (+ 870us) server_negotiation.cc:407] Connection header received
0812 03:07:49.325311 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:49.325314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:49.325374 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:49.325444 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:49.326565 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.327088 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.328215 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.328457 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.328549 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:49.332772 (+ 4223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:49.332775 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:49.332778 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:49.332801 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:49.334280 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.334779 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.334783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.334785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.334880 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.335178 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.335182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.335183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.335419 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:49.335549 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:49.336083 (+ 534us) server_negotiation.cc:299] Negotiation successful
0812 03:07:49.336216 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:07:49.336961 10611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.323805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:35993 (local address 127.0.0.1:46552)
0812 03:07:49.324239 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:49.324289 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:49.325099 (+ 810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:49.325610 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:49.325616 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:49.325948 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:49.326425 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:49.326439 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.327207 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.327210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:49.328099 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:49.328109 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.328592 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.328597 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:49.328756 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:49.329328 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:49.329344 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:49.332606 (+ 3262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:49.334411 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:49.334416 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:49.334427 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:49.334667 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:49.334965 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:49.334967 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:49.334968 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:49.335085 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:49.335526 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:49.335530 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:49.335819 (+ 289us) client_negotiation.cc:769] Sending connection context
0812 03:07:49.335976 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 03:07:49.336195 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":101,"threads_started":1}
I20260812 03:07:49.338371 10606 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.189" port: 37281 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:07:49 dist-test-slave-6bbx krb5kdc[10370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504069, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.189@KRBTEST.COM for kudu/127.5.96.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.96.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.188.keytab.
Entry for principal kudu/127.5.96.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.188.keytab.
I20260812 03:07:49.346416 10611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.338789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:37281 (local address 127.0.0.1:57290)
0812 03:07:49.338923 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:49.338934 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:49.338988 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:49.339523 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:49.339526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:49.339705 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:49.339792 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:49.339797 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.340711 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.340727 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:49.341280 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:49.341289 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.341815 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.341818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:49.341900 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:49.342277 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:49.342291 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:49.343422 (+ 1131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:49.345027 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:49.345030 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:49.345031 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:49.345208 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:49.345465 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:49.345467 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:49.345468 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:49.345530 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:49.345948 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:49.345951 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:49.346046 (+ 95us) client_negotiation.cc:769] Sending connection context
0812 03:07:49.346138 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 03:07:49.346271 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260812 03:07:49.346493 5506 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6aqATZ/build/debug/bin/kudu
/tmp/dist-test-task6aqATZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.188:39011
--webserver_interface=127.5.96.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.188.keytab
--principal=kudu/127.5.96.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.96.148:35503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.96.190:35993,127.5.96.189:37281,127.5.96.188:39011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:07:49.346539 10614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.338900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57290 (local address 127.5.96.189:37281)
0812 03:07:49.339216 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:49.339231 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:49.339248 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:07:49.339279 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:49.339282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:49.339414 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:49.339523 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:49.339885 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.340574 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.341381 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.341714 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.341804 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:49.343546 (+ 1742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:49.343554 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:49.343564 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:49.343585 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:49.344946 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.345318 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.345323 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.345324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.345400 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.345595 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.345598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.345599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.345854 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:49.345952 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:49.346241 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 03:07:49.346365 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":159,"threads_started":1}
I20260812 03:07:49.348477 10606 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:07:49.349090 10611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.348839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:39011 (local address 127.0.0.1:57886)
0812 03:07:49.348955 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:49.349035 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 03:07:49.349272 10538 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.96.188:39011: connect: Connection refused (error 111)
I20260812 03:07:49.350032 10611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.349850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:39011 (local address 127.0.0.1:57898)
0812 03:07:49.349941 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:49.349986 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 03:07:49.350149 10606 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.96.188:39011: Network error: Client connection negotiation failed: client connection to 127.5.96.188:39011: connect: Connection refused (error 111)
I20260812 03:07:49.358309 10502 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.96.189" port: 37281 } attempt: 3
Aug 12 03:07:49 dist-test-slave-6bbx krb5kdc[10370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504069, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for kudu/127.5.96.189@KRBTEST.COM
I20260812 03:07:49.365649 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.358717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:37281 (local address 127.0.0.1:57302)
0812 03:07:49.358852 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:49.358867 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:49.358956 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:49.359163 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:49.359167 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:49.359332 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:49.359421 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:49.359427 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.360172 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.360176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:49.360705 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:49.360713 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.361146 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.361150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:49.361262 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:49.361664 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:49.361678 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:49.362806 (+ 1128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:49.364472 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:49.364475 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:49.364476 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:49.364653 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:49.364903 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:49.364905 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:49.364906 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:49.364970 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:49.365297 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:49.365299 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:49.365375 (+ 76us) client_negotiation.cc:769] Sending connection context
0812 03:07:49.365443 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 03:07:49.365537 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 03:07:49.365758 10614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.358758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57302 (local address 127.5.96.189:37281)
0812 03:07:49.358860 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:49.358865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:49.358909 (+ 44us) server_negotiation.cc:407] Connection header received
0812 03:07:49.359033 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:49.359036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:49.359075 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:49.359135 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:49.359523 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.360074 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.360810 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.361049 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.361133 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:49.362933 (+ 1800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:49.362937 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:49.362939 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:49.362964 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:49.364373 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.364739 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.364743 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.364744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.364827 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.365035 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.365038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.365039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.365224 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:49.365314 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:49.365504 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 03:07:49.365608 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:07:49.366771 10502 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:07:49.367373 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.367174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:39011 (local address 127.0.0.1:57908)
0812 03:07:49.367271 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:49.367314 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 03:07:49.367995 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.367783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:39011 (local address 127.0.0.1:57920)
0812 03:07:49.367907 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:49.367947 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260812 03:07:49.368103 10502 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.96.188:39011: Network error: Client connection negotiation failed: client connection to 127.5.96.188:39011: connect: Connection refused (error 111)
I20260812 03:07:49.399250 10502 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 } attempt: 1
I20260812 03:07:49.399405 10606 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 } attempt: 1
I20260812 03:07:49.400043 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.399752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:39011 (local address 127.0.0.1:57934)
0812 03:07:49.399896 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:49.399961 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 03:07:49.400074 10611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.399846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:39011 (local address 127.0.0.1:57944)
0812 03:07:49.399962 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:49.400011 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 03:07:49.400847 10611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.400599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:39011 (local address 127.0.0.1:57952)
0812 03:07:49.400739 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:49.400784 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 03:07:49.400926 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.400613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:39011 (local address 127.0.0.1:57954)
0812 03:07:49.400713 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:49.400792 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 03:07:49.401023 10606 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.96.188:39011: Network error: Client connection negotiation failed: client connection to 127.5.96.188:39011: connect: Connection refused (error 111)
W20260812 03:07:49.401185 10502 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.96.188:39011: Network error: Client connection negotiation failed: client connection to 127.5.96.188:39011: connect: Connection refused (error 111)
W20260812 03:07:49.457674 10615 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:07:49.457928 10615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:49.457953 10615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:49.458019 10615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:49.461985 10615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:49.462033 10615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:49.462051 10615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:49.462080 10615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:49.462127 10615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:49.467104 10615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.96.148:35503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.96.190:35993,127.5.96.189:37281,127.5.96.188:39011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/krb5kdc/kudu_127.5.96.188.keytab
--principal=kudu/127.5.96.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.96.188:39011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.96.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:49.468050 10615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:49.469043 10615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:07:49.469188 10606 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 } attempt: 2
I20260812 03:07:49.469933 10611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.469655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:39011 (local address 127.0.0.1:57956)
0812 03:07:49.469801 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:49.469860 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 03:07:49.470682 10611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.470479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:39011 (local address 127.0.0.1:57962)
0812 03:07:49.470576 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:49.470623 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 03:07:49.470813 10606 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.96.188:39011: Network error: Client connection negotiation failed: client connection to 127.5.96.188:39011: connect: Connection refused (error 111)
I20260812 03:07:49.476359 10502 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 } attempt: 2
I20260812 03:07:49.477059 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.476755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:39011 (local address 127.0.0.1:57978)
0812 03:07:49.476892 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:49.476978 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 03:07:49.477254 10625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:49.477233 10624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:49.477298 10628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:49.477886 10615 server_base.cc:1034] running on GCE node
I20260812 03:07:49.478562 10615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:49.478669 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.478465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:39011 (local address 127.0.0.1:57992)
0812 03:07:49.478572 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:49.478616 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.96.188:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 03:07:49.478819 10502 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.96.188:39011: Network error: Client connection negotiation failed: client connection to 127.5.96.188:39011: connect: Connection refused (error 111)
I20260812 03:07:49.479624 10615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:49.480818 10615 hybrid_clock.cc:648] HybridClock initialized: now 1786504069480790 us; error 34 us; skew 500 ppm
Aug 12 03:07:49 dist-test-slave-6bbx krb5kdc[10370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504069, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:49.486017 10615 init.cc:377] Logged in from keytab as kudu/127.5.96.188@KRBTEST.COM (short username kudu)
I20260812 03:07:49.486845 10615 webserver.cc:460] Webserver started at http://127.5.96.188:33141/ using document root <none> and password file <none>
I20260812 03:07:49.487378 10615 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:49.487442 10615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:49.487635 10615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:49.489566 10615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-2/data/instance:
uuid: "249a14d81a5641d7a8cc7f024bb91c33"
format_stamp: "Formatted at 2026-08-12 03:07:49 on dist-test-slave-6bbx"
server_key: "0b3981856dcfcab35843f353342567c7"
server_key_iv: "9e6d3e3cde7858af0d327ea1754405ee"
server_key_version: "encryptionkey@0"
I20260812 03:07:49.490085 10615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-2/wal/instance:
uuid: "249a14d81a5641d7a8cc7f024bb91c33"
format_stamp: "Formatted at 2026-08-12 03:07:49 on dist-test-slave-6bbx"
server_key: "0b3981856dcfcab35843f353342567c7"
server_key_iv: "9e6d3e3cde7858af0d327ea1754405ee"
server_key_version: "encryptionkey@0"
I20260812 03:07:49.493548 10615 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260812 03:07:49.495733 10637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:49.496805 10615 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 03:07:49.496917 10615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-2/data,/tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-2/wal
uuid: "249a14d81a5641d7a8cc7f024bb91c33"
format_stamp: "Formatted at 2026-08-12 03:07:49 on dist-test-slave-6bbx"
server_key: "0b3981856dcfcab35843f353342567c7"
server_key_iv: "9e6d3e3cde7858af0d327ea1754405ee"
server_key_version: "encryptionkey@0"
I20260812 03:07:49.497000 10615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:49.514739 10615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:49.517243 10615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:49.517401 10615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:49.523733 10615 rpc_server.cc:307] RPC server started. Bound to: 127.5.96.188:39011
I20260812 03:07:49.523999 10713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.96.188:39011 every 8 connection(s)
I20260812 03:07:49.524618 10615 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-2/data/info.pb
I20260812 03:07:49.527153 10715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:49.530586 10715 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35993 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:07:49.532745 5506 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6aqATZ/build/debug/bin/kudu as pid 10615
I20260812 03:07:49.532868 5506 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6aqATZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786503790637001-5506-0/minicluster-data/master-2/wal/instance
I20260812 03:07:49.533128 5506 external_mini_cluster.cc:1428] Setting key 2113abaf47e5e0997269d9791e0f4ded
Aug 12 03:07:49 dist-test-slave-6bbx krb5kdc[10370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504069, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.188@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
Aug 12 03:07:49 dist-test-slave-6bbx krb5kdc[10370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.188@KRBTEST.COM
I20260812 03:07:49.543021 10509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.532211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46564 (local address 127.5.96.190:35993)
0812 03:07:49.532364 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:49.532368 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:49.533148 (+ 780us) server_negotiation.cc:407] Connection header received
0812 03:07:49.534257 (+ 1109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:49.534261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:49.534322 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:49.534390 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:49.535352 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.535893 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.536831 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.537037 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.537123 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:49.539507 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:49.539510 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:49.539512 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:49.539536 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:49.540974 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.541565 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.541567 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.541568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.541637 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.541936 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.541938 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.541939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.542164 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:49.542246 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:49.542766 (+ 520us) server_negotiation.cc:299] Negotiation successful
0812 03:07:49.542871 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 03:07:49.543813 10717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.532567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.190:35993 (local address 127.0.0.1:46564)
0812 03:07:49.532993 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:49.533043 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:49.533995 (+ 952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:49.534486 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:49.534496 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:49.534804 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:49.535199 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:49.535211 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.535999 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.536003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:49.536717 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:49.536727 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.537165 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.537168 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:49.537332 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:49.537866 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:49.537885 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:49.539372 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:49.541075 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:49.541082 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:49.541098 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:49.541479 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:49.541692 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:49.541695 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:49.541697 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:49.541855 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:49.542269 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:49.542275 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:49.542529 (+ 254us) client_negotiation.cc:769] Sending connection context
0812 03:07:49.542687 (+ 158us) client_negotiation.cc:240] Negotiation successful
0812 03:07:49.542965 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":96,"threads_started":1}
I20260812 03:07:49.543816 10719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.533739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57994 (local address 127.5.96.188:39011)
0812 03:07:49.534103 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:49.534117 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:49.534158 (+ 41us) server_negotiation.cc:407] Connection header received
0812 03:07:49.534433 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:49.534437 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:49.534602 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:49.534756 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:49.535525 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.536376 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.537178 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.537516 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.537620 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:49.539945 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:49.539949 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:49.539958 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:49.539979 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:49.541573 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.542008 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.542014 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.542016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.542114 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.542384 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.542386 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.542387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.542592 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:49.542778 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:49.542996 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 03:07:49.543127 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"spinlock_wait_cycles":768,"thread_start_us":116,"threads_started":1}
I20260812 03:07:49.544970 10715 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.189" port: 37281 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:07:49 dist-test-slave-6bbx krb5kdc[10370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504069, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.188@KRBTEST.COM for kudu/127.5.96.189@KRBTEST.COM
Aug 12 03:07:49 dist-test-slave-6bbx krb5kdc[10370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.190@KRBTEST.COM
I20260812 03:07:49.552465 10717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.545374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.189:37281 (local address 127.0.0.1:57316)
0812 03:07:49.545504 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:49.545514 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:49.545570 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:49.545872 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:49.545875 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:49.546008 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:49.546086 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:49.546090 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.546768 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.546771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:49.547238 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:49.547245 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.547643 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.547646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:49.547763 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:49.548137 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:49.548148 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:49.549399 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:49.551060 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:49.551064 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:49.551066 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:49.551310 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:49.551571 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:49.551573 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:49.551574 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:49.551642 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:49.552038 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:49.552040 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:49.552133 (+ 93us) client_negotiation.cc:769] Sending connection context
0812 03:07:49.552236 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 03:07:49.552346 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 03:07:49.552533 10614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.545521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57316 (local address 127.5.96.189:37281)
0812 03:07:49.545660 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:49.545665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:49.545678 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:07:49.545718 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:49.545721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:49.545767 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:49.545835 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:49.546196 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.546665 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.547341 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.547552 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.547636 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:49.549524 (+ 1888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:49.549527 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:49.549529 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:49.549551 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:49.550961 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.551398 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.551401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.551403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.551490 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.551733 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.551737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.551738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.551948 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:49.552052 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:49.552296 (+ 244us) server_negotiation.cc:299] Negotiation successful
0812 03:07:49.552391 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 03:07:49.553081 10715 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:07:49.553591 10509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.545817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46580 (local address 127.5.96.190:35993)
0812 03:07:49.545911 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:49.545915 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:49.545984 (+ 69us) server_negotiation.cc:407] Connection header received
0812 03:07:49.546098 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:49.546100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:49.546137 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:49.546195 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:49.546729 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.547169 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.547938 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.548147 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.548237 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:49.550432 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:49.550436 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:49.550439 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:49.550466 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:49.551970 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.552405 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.552408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.552409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.552484 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.552793 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.552796 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.552798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.553008 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:49.553204 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:49.553350 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 03:07:49.553448 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
Aug 12 03:07:49 dist-test-slave-6bbx krb5kdc[10370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504069, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.188@KRBTEST.COM for kudu/127.5.96.188@KRBTEST.COM
Aug 12 03:07:49 dist-test-slave-6bbx krb5kdc[10370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.96.189@KRBTEST.COM
I20260812 03:07:49.560469 10717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.553458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:39011 (local address 127.0.0.1:58010)
0812 03:07:49.553535 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:49.553547 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:49.553611 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:49.553822 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:49.553825 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:49.553973 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:49.554047 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:49.554053 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.554804 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.554808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:49.555531 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:49.555540 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.556103 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.556105 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:49.556180 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:49.556523 (+ 343us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:49.556537 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:49.557686 (+ 1149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:49.559118 (+ 1432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:49.559120 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:49.559122 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:49.559294 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:49.559563 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:49.559567 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:49.559568 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:49.559696 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:49.560013 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:49.560016 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:49.560092 (+ 76us) client_negotiation.cc:769] Sending connection context
0812 03:07:49.560189 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 03:07:49.560304 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":1}
I20260812 03:07:49.560563 10719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.553579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58010 (local address 127.5.96.188:39011)
0812 03:07:49.553663 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:49.553667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:49.553677 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:07:49.553702 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:49.553705 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:49.553735 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:49.553799 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:49.554163 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.554721 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.555630 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.556003 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.556102 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:49.557807 (+ 1705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:49.557810 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:49.557812 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:49.557833 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:49.559014 (+ 1181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.559385 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.559388 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.559390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.559466 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.559777 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.559779 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.559780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.559923 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:49.560006 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:49.560264 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 03:07:49.560367 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":33}
I20260812 03:07:49.561965 10606 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 } attempt: 3
I20260812 03:07:49.563206 10614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.554605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57318 (local address 127.5.96.189:37281)
0812 03:07:49.554702 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:49.554706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:49.554719 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:07:49.554821 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:49.554824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:49.554865 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:49.554920 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:49.555556 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.556244 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.557199 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.557416 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.557506 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:49.559941 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:49.559944 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:49.559946 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:49.559966 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:49.561566 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.561938 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.561941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.561942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.562010 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.562277 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.562280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.562282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.562655 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:49.562901 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:49.562976 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 03:07:49.563074 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 03:07:49.563933 10715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 249a14d81a5641d7a8cc7f024bb91c33: Bootstrap starting.
I20260812 03:07:49.566231 5506 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:07:49.566695 10715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 249a14d81a5641d7a8cc7f024bb91c33: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:49.567440 10715 log.cc:824] T 00000000000000000000000000000000 P 249a14d81a5641d7a8cc7f024bb91c33: Log is configured to *not* fsync() on all Append() calls
Aug 12 03:07:49 dist-test-slave-6bbx krb5kdc[10370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504069, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.189@KRBTEST.COM for kudu/127.5.96.188@KRBTEST.COM
I20260812 03:07:49.567370 10456 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2fb47476959a44518470346eec9b49e4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46580
I20260812 03:07:49.569490 10715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 249a14d81a5641d7a8cc7f024bb91c33: No bootstrap required, opened a new log
I20260812 03:07:49.571688 10611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.562364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:39011 (local address 127.0.0.1:58018)
0812 03:07:49.563825 (+ 1461us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:49.563837 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:49.563910 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:49.564110 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:49.564113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:49.564267 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:49.564351 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:49.564358 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.565034 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.565037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:49.565669 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:49.565681 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.566535 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.566539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:49.566621 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:49.567138 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:49.567152 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:49.568517 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:49.570241 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:49.570244 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:49.570246 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:49.570482 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:49.570745 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:49.570748 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:49.570750 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:49.570829 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:49.571212 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:49.571214 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:49.571306 (+ 92us) client_negotiation.cc:769] Sending connection context
0812 03:07:49.571397 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 03:07:49.571514 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1386}
I20260812 03:07:49.571727 10719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.562452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58018 (local address 127.5.96.188:39011)
0812 03:07:49.562551 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:49.562554 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:49.563892 (+ 1338us) server_negotiation.cc:407] Connection header received
0812 03:07:49.563974 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:49.563977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:49.564016 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:49.564084 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:49.564461 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.564933 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.565772 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.566417 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.566519 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:49.568666 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:49.568670 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:49.568672 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:49.568696 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:49.570133 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.570579 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.570582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.570584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.570662 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.570905 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.570907 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.570909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.571124 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:49.571211 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:49.571459 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 03:07:49.571561 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:07:49.573276 10715 raft_consensus.cc:348] T 00000000000000000000000000000000 P 249a14d81a5641d7a8cc7f024bb91c33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb47476959a44518470346eec9b49e4" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35993 } } peers { permanent_uuid: "72920dc6e86f4a26bdf93656ee298281" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 37281 } } peers { permanent_uuid: "249a14d81a5641d7a8cc7f024bb91c33" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 } }
I20260812 03:07:49.573498 10715 raft_consensus.cc:729] T 00000000000000000000000000000000 P 249a14d81a5641d7a8cc7f024bb91c33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 249a14d81a5641d7a8cc7f024bb91c33, State: Initialized, Role: FOLLOWER
I20260812 03:07:49.573920 10715 consensus_queue.cc:260] T 00000000000000000000000000000000 P 249a14d81a5641d7a8cc7f024bb91c33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb47476959a44518470346eec9b49e4" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35993 } } peers { permanent_uuid: "72920dc6e86f4a26bdf93656ee298281" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 37281 } } peers { permanent_uuid: "249a14d81a5641d7a8cc7f024bb91c33" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 } }
I20260812 03:07:49.574182 10606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72920dc6e86f4a26bdf93656ee298281: Bootstrap starting.
I20260812 03:07:49.574390 10721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 249a14d81a5641d7a8cc7f024bb91c33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb47476959a44518470346eec9b49e4" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35993 } } peers { permanent_uuid: "72920dc6e86f4a26bdf93656ee298281" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 37281 } } peers { permanent_uuid: "249a14d81a5641d7a8cc7f024bb91c33" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 } } }
I20260812 03:07:49.574527 10721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 249a14d81a5641d7a8cc7f024bb91c33 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:07:49.575212 10715 sys_catalog.cc:564] T 00000000000000000000000000000000 P 249a14d81a5641d7a8cc7f024bb91c33 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:49.576395 10606 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72920dc6e86f4a26bdf93656ee298281: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:49.577096 10606 log.cc:824] T 00000000000000000000000000000000 P 72920dc6e86f4a26bdf93656ee298281: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:49.577921 10502 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 } attempt: 3
W20260812 03:07:49.580030 10740 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 249a14d81a5641d7a8cc7f024bb91c33: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:07:49.580114 10740 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:07:49.578751 10615 master_runner.cc:186] Error getting master registration for 127.5.96.190:35993: OK, instance_id { permanent_uuid: "2fb47476959a44518470346eec9b49e4" instance_seqno: 1786504069119725 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:07:49.582399 10615 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 12 03:07:49 dist-test-slave-6bbx krb5kdc[10370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504069, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.96.190@KRBTEST.COM for kudu/127.5.96.188@KRBTEST.COM
I20260812 03:07:49.585657 10719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.578480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58030 (local address 127.5.96.188:39011)
0812 03:07:49.578569 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:49.578573 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:49.578583 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:07:49.578613 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:49.578615 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:49.578649 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:49.578714 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:49.579093 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.579863 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.580618 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.580944 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.581028 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:49.583024 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:49.583027 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:49.583030 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:49.583052 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:49.584263 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.584655 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.584658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.584660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.584729 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.584967 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.584970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.584971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.585121 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:49.585200 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:49.585442 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 03:07:49.585531 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 03:07:49.585659 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.578279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.96.188:39011 (local address 127.0.0.1:58030)
0812 03:07:49.578383 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:49.578396 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:49.578470 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:49.578742 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:49.578746 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:49.578907 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:49.578987 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:49.578993 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.579959 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.579962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:49.580513 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:49.580522 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.581040 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.581044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:49.581142 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:49.581550 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:49.581567 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:49.582898 (+ 1331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:49.584360 (+ 1462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:49.584363 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:49.584365 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:49.584569 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:49.584808 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:49.584811 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:49.584813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:49.584891 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:49.585207 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:49.585210 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:49.585299 (+ 89us) client_negotiation.cc:769] Sending connection context
0812 03:07:49.585384 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 03:07:49.585505 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 03:07:49.588435 10502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4: Bootstrap starting.
I20260812 03:07:49.590375 10502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:49.591017 10502 log.cc:824] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:49.592873 10502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4: No bootstrap required, opened a new log
I20260812 03:07:49.593317 10606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72920dc6e86f4a26bdf93656ee298281: No bootstrap required, opened a new log
I20260812 03:07:49.595743 10502 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb47476959a44518470346eec9b49e4" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35993 } } peers { permanent_uuid: "72920dc6e86f4a26bdf93656ee298281" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 37281 } } peers { permanent_uuid: "249a14d81a5641d7a8cc7f024bb91c33" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 } }
I20260812 03:07:49.595958 10502 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fb47476959a44518470346eec9b49e4, State: Initialized, Role: FOLLOWER
I20260812 03:07:49.596271 10502 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb47476959a44518470346eec9b49e4" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35993 } } peers { permanent_uuid: "72920dc6e86f4a26bdf93656ee298281" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 37281 } } peers { permanent_uuid: "249a14d81a5641d7a8cc7f024bb91c33" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 } }
I20260812 03:07:49.596875 10747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb47476959a44518470346eec9b49e4" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35993 } } peers { permanent_uuid: "72920dc6e86f4a26bdf93656ee298281" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 37281 } } peers { permanent_uuid: "249a14d81a5641d7a8cc7f024bb91c33" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 } } }
I20260812 03:07:49.597134 10747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:07:49.597067 10606 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72920dc6e86f4a26bdf93656ee298281 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb47476959a44518470346eec9b49e4" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35993 } } peers { permanent_uuid: "72920dc6e86f4a26bdf93656ee298281" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 37281 } } peers { permanent_uuid: "249a14d81a5641d7a8cc7f024bb91c33" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 } }
I20260812 03:07:49.597289 10606 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72920dc6e86f4a26bdf93656ee298281 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72920dc6e86f4a26bdf93656ee298281, State: Initialized, Role: FOLLOWER
I20260812 03:07:49.597527 10502 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:49.597620 10456 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 03:07:49.597584 10606 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72920dc6e86f4a26bdf93656ee298281 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb47476959a44518470346eec9b49e4" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35993 } } peers { permanent_uuid: "72920dc6e86f4a26bdf93656ee298281" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 37281 } } peers { permanent_uuid: "249a14d81a5641d7a8cc7f024bb91c33" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 } }
I20260812 03:07:49.597729 10456 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:49.598857 10606 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72920dc6e86f4a26bdf93656ee298281 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:49.598847 10456 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb47476959a44518470346eec9b49e4" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35993 } } peers { permanent_uuid: "72920dc6e86f4a26bdf93656ee298281" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 37281 } } peers { permanent_uuid: "249a14d81a5641d7a8cc7f024bb91c33" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 } }
I20260812 03:07:49.599716 10456 leader_election.cc:288] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4 [CANDIDATE]: Term 1 election: Requested vote from peers 72920dc6e86f4a26bdf93656ee298281 (127.5.96.189:37281), 249a14d81a5641d7a8cc7f024bb91c33 (127.5.96.188:39011)
I20260812 03:07:49.601308 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2fb47476959a44518470346eec9b49e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "249a14d81a5641d7a8cc7f024bb91c33"
I20260812 03:07:49.601534 10679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 249a14d81a5641d7a8cc7f024bb91c33 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:49.602767 10679 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 249a14d81a5641d7a8cc7f024bb91c33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fb47476959a44518470346eec9b49e4 in term 1.
W20260812 03:07:49.604036 10763 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:07:49.604100 10763 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:07:49.605224 10426 leader_election.cc:302] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 249a14d81a5641d7a8cc7f024bb91c33, 2fb47476959a44518470346eec9b49e4; no voters:
I20260812 03:07:49.600937 10571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2fb47476959a44518470346eec9b49e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "72920dc6e86f4a26bdf93656ee298281"
I20260812 03:07:49.606798 10571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72920dc6e86f4a26bdf93656ee298281 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:49.608009 10571 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 72920dc6e86f4a26bdf93656ee298281 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fb47476959a44518470346eec9b49e4 in term 1.
W20260812 03:07:49.603504 10781 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 72920dc6e86f4a26bdf93656ee298281: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:07:49.608196 10781 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:07:49.613402 10392 master_runner.cc:263] No leader master found from master 2fb47476959a44518470346eec9b49e4
I20260812 03:07:49.613471 10392 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:07:49.613430 10747 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:49.613737 10747 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4 [term 1 LEADER]: Becoming Leader. State: Replica: 2fb47476959a44518470346eec9b49e4, State: Running, Role: LEADER
I20260812 03:07:49.614075 10747 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb47476959a44518470346eec9b49e4" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35993 } } peers { permanent_uuid: "72920dc6e86f4a26bdf93656ee298281" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 37281 } } peers { permanent_uuid: "249a14d81a5641d7a8cc7f024bb91c33" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 } }
I20260812 03:07:49.615525 10515 master_runner.cc:263] No leader master found from master 72920dc6e86f4a26bdf93656ee298281
I20260812 03:07:49.615592 10515 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:07:49.615945 10747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fb47476959a44518470346eec9b49e4. Latest consensus state: current_term: 1 leader_uuid: "2fb47476959a44518470346eec9b49e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb47476959a44518470346eec9b49e4" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35993 } } peers { permanent_uuid: "72920dc6e86f4a26bdf93656ee298281" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 37281 } } peers { permanent_uuid: "249a14d81a5641d7a8cc7f024bb91c33" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 } } }
W20260812 03:07:49.616467 10429 tablet.cc:2369] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:07:49.616498 10747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:49.617426 10796 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:49.618326 10614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.604842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57330 (local address 127.5.96.189:37281)
0812 03:07:49.604941 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:49.604945 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:49.606012 (+ 1067us) server_negotiation.cc:407] Connection header received
0812 03:07:49.606185 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:49.606189 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:49.606231 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:49.606293 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:49.607304 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.608052 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.609214 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.609552 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.609628 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:49.611357 (+ 1729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:49.611361 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:49.611364 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:49.611389 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:49.612939 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.616940 (+ 4001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.616944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.616946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.617036 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.617372 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.617375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.617377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.617583 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:49.617686 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:49.618050 (+ 364us) server_negotiation.cc:299] Negotiation successful
0812 03:07:49.618162 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 03:07:49.619783 10748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72920dc6e86f4a26bdf93656ee298281 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb47476959a44518470346eec9b49e4" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35993 } } peers { permanent_uuid: "72920dc6e86f4a26bdf93656ee298281" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 37281 } } peers { permanent_uuid: "249a14d81a5641d7a8cc7f024bb91c33" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 } } }
I20260812 03:07:49.619902 10748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72920dc6e86f4a26bdf93656ee298281 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:07:49.622466 10796 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:49.638736 10719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.605438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58042 (local address 127.5.96.188:39011)
0812 03:07:49.605534 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:49.605537 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:49.617207 (+ 11670us) server_negotiation.cc:407] Connection header received
0812 03:07:49.618622 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:49.618625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:49.618672 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:49.618719 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:49.622616 (+ 3897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.623360 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.628306 (+ 4946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.628640 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.628706 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:49.632349 (+ 3643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:49.632352 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:49.632355 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:49.632380 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:49.633891 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.636442 (+ 2551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.636446 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.636447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.636534 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.637198 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.637201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.637203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.637406 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:49.637522 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:49.638456 (+ 934us) server_negotiation.cc:299] Negotiation successful
0812 03:07:49.638569 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 03:07:49.640188 10571 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 72920dc6e86f4a26bdf93656ee298281 [term 1 FOLLOWER]: Refusing update from remote peer 2fb47476959a44518470346eec9b49e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:07:49.640406 10679 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 249a14d81a5641d7a8cc7f024bb91c33 [term 1 FOLLOWER]: Refusing update from remote peer 2fb47476959a44518470346eec9b49e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:07:49.640875 10792 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72920dc6e86f4a26bdf93656ee298281" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 37281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:07:49.641234 10747 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "249a14d81a5641d7a8cc7f024bb91c33" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:07:49.644663 10748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72920dc6e86f4a26bdf93656ee298281 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fb47476959a44518470346eec9b49e4. Latest consensus state: current_term: 1 leader_uuid: "2fb47476959a44518470346eec9b49e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb47476959a44518470346eec9b49e4" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35993 } } peers { permanent_uuid: "72920dc6e86f4a26bdf93656ee298281" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 37281 } } peers { permanent_uuid: "249a14d81a5641d7a8cc7f024bb91c33" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 } } }
I20260812 03:07:49.644773 10748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72920dc6e86f4a26bdf93656ee298281 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:07:49.645109 10721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 249a14d81a5641d7a8cc7f024bb91c33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fb47476959a44518470346eec9b49e4. Latest consensus state: current_term: 1 leader_uuid: "2fb47476959a44518470346eec9b49e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb47476959a44518470346eec9b49e4" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35993 } } peers { permanent_uuid: "72920dc6e86f4a26bdf93656ee298281" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 37281 } } peers { permanent_uuid: "249a14d81a5641d7a8cc7f024bb91c33" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 } } }
I20260812 03:07:49.645207 10721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 249a14d81a5641d7a8cc7f024bb91c33 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:07:49.645713 10509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:49.616373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46596 (local address 127.5.96.190:35993)
0812 03:07:49.616468 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:49.616472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:49.618959 (+ 2487us) server_negotiation.cc:407] Connection header received
0812 03:07:49.619888 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:49.619892 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:49.619933 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:49.619977 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:49.626401 (+ 6424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.627150 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.629303 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:49.629636 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:49.629704 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:49.634379 (+ 4675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:49.634383 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:49.634386 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:49.634411 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:49.635977 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.636834 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.636837 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.636839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.636919 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:49.637817 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:49.637820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:49.637822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:49.638022 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:49.638132 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:49.645456 (+ 7324us) server_negotiation.cc:299] Negotiation successful
0812 03:07:49.645570 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 03:07:49.654167 10792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2fb47476959a44518470346eec9b49e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb47476959a44518470346eec9b49e4" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35993 } } peers { permanent_uuid: "72920dc6e86f4a26bdf93656ee298281" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 37281 } } peers { permanent_uuid: "249a14d81a5641d7a8cc7f024bb91c33" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 } } }
I20260812 03:07:49.654305 10792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:49.654402 10799 mvcc.cc:205] Tried to move back new op lower bound from 7317520669218217984 to 7317520669142880256. Current Snapshot: MvccSnapshot[applied={T|T < 7317520669218217984}]
I20260812 03:07:49.654631 10721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 249a14d81a5641d7a8cc7f024bb91c33 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2fb47476959a44518470346eec9b49e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb47476959a44518470346eec9b49e4" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35993 } } peers { permanent_uuid: "72920dc6e86f4a26bdf93656ee298281" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 37281 } } peers { permanent_uuid: "249a14d81a5641d7a8cc7f024bb91c33" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 } } }
I20260812 03:07:49.654716 10721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 249a14d81a5641d7a8cc7f024bb91c33 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:07:49.655117 10801 mvcc.cc:205] Tried to move back new op lower bound from 7317520669218217984 to 7317520669142880256. Current Snapshot: MvccSnapshot[applied={T|T < 7317520669218217984}]
I20260812 03:07:49.656008 10747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2fb47476959a44518470346eec9b49e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb47476959a44518470346eec9b49e4" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35993 } } peers { permanent_uuid: "72920dc6e86f4a26bdf93656ee298281" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 37281 } } peers { permanent_uuid: "249a14d81a5641d7a8cc7f024bb91c33" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 } } }
I20260812 03:07:49.656128 10747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:49.657130 10748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72920dc6e86f4a26bdf93656ee298281 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2fb47476959a44518470346eec9b49e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb47476959a44518470346eec9b49e4" member_type: VOTER last_known_addr { host: "127.5.96.190" port: 35993 } } peers { permanent_uuid: "72920dc6e86f4a26bdf93656ee298281" member_type: VOTER last_known_addr { host: "127.5.96.189" port: 37281 } } peers { permanent_uuid: "249a14d81a5641d7a8cc7f024bb91c33" member_type: VOTER last_known_addr { host: "127.5.96.188" port: 39011 } } }
I20260812 03:07:49.657245 10748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72920dc6e86f4a26bdf93656ee298281 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:07:49.659955 10796 catalog_manager.cc:1347] Generated new cluster ID: 6883dddbe79f4bf0933a9343e6e7aef9
I20260812 03:07:49.660009 10796 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:49.680413 10796 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:49.682130 10796 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:49.697098 10796 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2fb47476959a44518470346eec9b49e4: Generated new TSK 0
I20260812 03:07:49.697844 10796 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:07:49.725380 10614 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:07:49.722450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57334 (local address 127.5.96.189:37281)
0812 03:07:49.722559 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:49.722563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:49.724297 (+ 1734us) server_negotiation.cc:407] Connection header received
0812 03:07:49.724562 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:49.724565 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:49.724613 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:49.724662 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:49.725244 (+ 582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57334: BlockingRecv error: recv got EOF from 127.0.0.1:57334 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260812 03:07:49.726946 10719 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:07:49.723072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58058 (local address 127.5.96.188:39011)
0812 03:07:49.723172 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:49.723176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:49.725733 (+ 2557us) server_negotiation.cc:407] Connection header received
0812 03:07:49.726203 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:49.726207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:49.726262 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:49.726309 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:49.726837 (+ 528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58058: BlockingRecv error: recv got EOF from 127.0.0.1:58058 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260812 03:07:49.727975 10509 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:07:49.723694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46602 (local address 127.5.96.190:35993)
0812 03:07:49.723802 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:49.723806 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:49.726031 (+ 2225us) server_negotiation.cc:407] Connection header received
0812 03:07:49.727277 (+ 1246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:49.727280 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:49.727327 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:49.727375 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:49.727862 (+ 487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46602: BlockingRecv error: recv got EOF from 127.0.0.1:46602 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260812 03:07:49.732141 10614 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:07:49.730709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57340 (local address 127.5.96.189:37281)
0812 03:07:49.730802 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:49.730805 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:49.731243 (+ 438us) server_negotiation.cc:407] Connection header received
0812 03:07:49.731455 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:49.731458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:49.731523 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:49.731572 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:49.732053 (+ 481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57340: BlockingRecv error: recv got EOF from 127.0.0.1:57340 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260812 03:07:49.736689 10614 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:07:49.735266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57354 (local address 127.5.96.189:37281)
0812 03:07:49.735360 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:49.735364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:49.735882 (+ 518us) server_negotiation.cc:407] Connection header received
0812 03:07:49.736039 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:49.736042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:49.736085 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:49.736132 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:49.736604 (+ 472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57354: BlockingRecv error: recv got EOF from 127.0.0.1:57354 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260812 03:07:49.741878 10614 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:07:49.740468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57366 (local address 127.5.96.189:37281)
0812 03:07:49.740558 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:49.740562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:49.741066 (+ 504us) server_negotiation.cc:407] Connection header received
0812 03:07:49.741244 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:49.741257 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:49.741305 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:49.741352 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:49.741794 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57366: BlockingRecv error: recv got EOF from 127.0.0.1:57366 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260812 03:07:49.749080 10614 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:07:49.747641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57382 (local address 127.5.96.189:37281)
0812 03:07:49.747787 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:49.747791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:49.748301 (+ 510us) server_negotiation.cc:407] Connection header received
0812 03:07:49.748468 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:49.748471 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:49.748514 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:49.748558 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:49.748997 (+ 439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57382: BlockingRecv error: recv got EOF from 127.0.0.1:57382 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260812 03:07:49.760373 10614 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:07:49.758880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57398 (local address 127.5.96.189:37281)
0812 03:07:49.758971 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:49.758974 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:49.759483 (+ 509us) server_negotiation.cc:407] Connection header received
0812 03:07:49.759657 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:49.759660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:49.759741 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:49.759794 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:49.760292 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57398: BlockingRecv error: recv got EOF from 127.0.0.1:57398 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260812 03:07:49.780061 10614 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:07:49.778473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57400 (local address 127.5.96.189:37281)
0812 03:07:49.778578 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:49.778583 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:49.779137 (+ 554us) server_negotiation.cc:407] Connection header received
0812 03:07:49.779329 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:49.779332 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:49.779379 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:49.779427 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:49.779954 (+ 527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57400: BlockingRecv error: recv got EOF from 127.0.0.1:57400 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260812 03:07:49.815204 10614 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:07:49.814155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57410 (local address 127.5.96.189:37281)
0812 03:07:49.814283 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:49.814288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:49.814420 (+ 132us) server_negotiation.cc:407] Connection header received
0812 03:07:49.814538 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:49.814542 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:49.814598 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:49.814696 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:49.815087 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57410: BlockingRecv error: recv got EOF from 127.0.0.1:57410 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260812 03:07:49.882301 10614 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:07:49.881158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57426 (local address 127.5.96.189:37281)
0812 03:07:49.881311 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:49.881317 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:49.881446 (+ 129us) server_negotiation.cc:407] Connection header received
0812 03:07:49.881607 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:49.881610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:49.881658 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:49.881722 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:49.882178 (+ 456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57426: BlockingRecv error: recv got EOF from 127.0.0.1:57426 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260812 03:07:50.013355 10614 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:07:50.012182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57438 (local address 127.5.96.189:37281)
0812 03:07:50.012324 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:50.012329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:50.012505 (+ 176us) server_negotiation.cc:407] Connection header received
0812 03:07:50.012643 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:50.012646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:50.012697 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:50.012766 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:50.013244 (+ 478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57438: BlockingRecv error: recv got EOF from 127.0.0.1:57438 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260812 03:07:50.272305 10614 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:07:50.271222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57450 (local address 127.5.96.189:37281)
0812 03:07:50.271354 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:50.271359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:50.271441 (+ 82us) server_negotiation.cc:407] Connection header received
0812 03:07:50.271583 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:50.271586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:50.271635 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:50.271729 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:50.272192 (+ 463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57450: BlockingRecv error: recv got EOF from 127.0.0.1:57450 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:07:50.581405 10740 catalog_manager.cc:1259] Loaded cluster ID: 6883dddbe79f4bf0933a9343e6e7aef9
I20260812 03:07:50.581492 10740 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 249a14d81a5641d7a8cc7f024bb91c33: loading cluster ID for follower catalog manager: success
I20260812 03:07:50.583192 10740 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 249a14d81a5641d7a8cc7f024bb91c33: acquiring CA information for follower catalog manager: success
I20260812 03:07:50.584393 10740 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 249a14d81a5641d7a8cc7f024bb91c33: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:07:50.609346 10781 catalog_manager.cc:1259] Loaded cluster ID: 6883dddbe79f4bf0933a9343e6e7aef9
I20260812 03:07:50.609408 10781 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 72920dc6e86f4a26bdf93656ee298281: loading cluster ID for follower catalog manager: success
I20260812 03:07:50.611032 10781 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 72920dc6e86f4a26bdf93656ee298281: acquiring CA information for follower catalog manager: success
I20260812 03:07:50.612179 10781 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 72920dc6e86f4a26bdf93656ee298281: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:07:50.790244 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:50.786377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57464 (local address 127.5.96.189:37281)
0812 03:07:50.786637 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:50.786642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:50.786658 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:07:50.786812 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:50.786816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:50.786949 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:50.787049 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:07:50.787338 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:50.787886 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:50.788746 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:50.788974 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:50.789065 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:50.789385 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:07:50.789843 (+ 458us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:07:50.789957 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:50.790025 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 03:07:50.790086 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":79,"threads_started":1}
I20260812 03:07:50.796051 10926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:50.792958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58074 (local address 127.5.96.188:39011)
0812 03:07:50.793151 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:50.793154 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:50.793168 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:07:50.793286 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:50.793289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:50.793399 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:50.793493 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:07:50.793647 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:50.794132 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:50.794865 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:50.795065 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:50.795129 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:50.795333 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:07:50.795731 (+ 398us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:07:50.795817 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:50.795863 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 03:07:50.795908 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":61,"threads_started":1}
I20260812 03:07:50.802678 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:50.798556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57476 (local address 127.5.96.189:37281)
0812 03:07:50.798662 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:50.798666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:50.799049 (+ 383us) server_negotiation.cc:407] Connection header received
0812 03:07:50.799144 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:50.799147 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:50.799192 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:50.799269 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:07:50.799466 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:50.800224 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:50.801547 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:50.801882 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:50.801972 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:50.802278 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:07:50.802353 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:07:50.802412 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:50.802516 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 03:07:50.802562 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 03:07:50.803448 10937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:50.798880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46612 (local address 127.5.96.190:35993)
0812 03:07:50.799293 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:50.799297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:50.799310 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:07:50.799389 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:50.799391 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:50.799522 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:50.799626 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:07:50.799939 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:50.800730 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:50.801861 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:50.802103 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:50.802235 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:50.802610 (+ 375us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:07:50.802914 (+ 304us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:07:50.802995 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:50.803270 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 03:07:50.803333 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":79,"threads_started":1}
I20260812 03:07:50.803845 10926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:50.799225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58090 (local address 127.5.96.188:39011)
0812 03:07:50.799534 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:50.799538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:50.800000 (+ 462us) server_negotiation.cc:407] Connection header received
0812 03:07:50.800129 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:50.800132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:50.800178 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:50.800246 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:07:50.800480 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:50.801049 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:50.802240 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:50.802535 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:50.802621 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:50.803138 (+ 517us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:07:50.803206 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:07:50.803287 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:50.803696 (+ 409us) server_negotiation.cc:299] Negotiation successful
0812 03:07:50.803744 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226}
I20260812 03:07:50.806178 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 10392
I20260812 03:07:50.824242 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 10515
I20260812 03:07:50.836334 5506 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6aqATZ/build/debug/bin/kudu with pid 10615
2026-08-12T03:07:50Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3141 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6315 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (280160 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST